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FOREWORD 


As  the  Bureau  of  Quality  Assurance  began  implementation  of  the  PSRO  program 
in  hospitals,  it  soon  became  evident  that  the  future  would  require  PSROs  to 
be  actively  involved  in  ambulatory  medical  care  quality  assurance.  Since 
the  state  of  the  art  of  ambulatory  care  review  is  far  less  advanced  than 
that  of  hospital  review,  we  realized  that  it  would  be  necessary  to  develop 
a  sound  basis  for  the  careful,  gradual  movement  of  PSROs  into  ambulatory 
care  quality  assurance. 

This  three-volume  report  is  the  result  of  the  work  completed  by  Health  Care 
Management  Systems,  Inc.,  in  one  of  the  first  major  Bureau  contracts  to 
acquire  and  document  knowledge  of  presently  operational  ambulatory  care 
quality  assurance  projects.    The  purpose  of  this  project  was  to  systematically 
assess  and  document  existing  activities  in  a  variety  of  health  delivery  set- 
tings across  the  country.    During  the  course  of  the  project,  a  generic 
quality  assurance  model  was  developed  for  use  in  the  design,  implementation, 
and  assessment  of  ambulatory  care  review  systems.    This  model  is  now  under 
refinement  and  is  being  applied  in  the  Bureau's  cooperative  ambulatory  care 
quality  assurance  demonstration  project.    It  represents  a  significant 
advance  in  our  ability  to  design  and  assess  such  systems. 

This  three-part  report,  which  includes  a  description  of  the  generic  model 
and  study  findings,  description  of  each  of  the  27  ambulatory  quality 
assurance  systems  included  in  the  survey,  and  a  bibliographic  index  that 
has  been  enriched  by  selected  abstracts,  should  provide  the  reader  with  a 
concise  overview  of  the  state  of  the  art.    The  results  of  this  project  are 
an  important  part  of  the  foundation  upon  which  to  build  well-balanced  and 
meaningful  PSRO  participation  in  ambulatory  care  quality  assurance.  The 
report  also  reflects  the  investigators'  concern  with  ambulatory  quality 
assurance  outside  the  scope  of  PSRO. 

The  authors  of  the  report  developed  their  material  in  a  relatively  short 
time  to  meet  the  need  for  fundamental  knowledge  of  existing  quality 
assurance  programs  to  support  the  implementation  of  voluntary  systems  and 
governmental  regulatory  programs.    It  is  hoped  that  any  limitations  in 
the  material  presented  herein  will  be  corrected  through  additional  research 
and  careful  experimentation  and  demonstration  of  new  ambulatory  quality 
assurance  methods  and  organization. 

We  believe  that  this  report  will  be  valuable  to  those  within  and  outside 
the  Federal  Government  who  are  interested  in  establishing  effective  quality 
assurance  systems  in  ambulatory  medical  care.    We  are  pleased  to  have 
provided  partial  funding  to  the  project  and  the  preparation  of  this  mono- 
graph and  hope  that  it  will  be  useful  to  all  who  are  now  engaged  in 
activities  leading  to  improved  quality  in  ambulatory  medical  care. 


Michael  J.  Goran,  M.D. 
Director 

Bureau  of  Quality  Assurance 
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ABSTRACT 


That  quality  care  assurance  is  in  a  state  of  flux  is  attested  to  by 
current  literature  in  the  field  and  is  substantiated  by  the  Ambulatory 
Care  Quality  Assurance  Project  (ACQAP)  report.    This  nationwide  survey 
was  undertaken  to  reveal  the  state  of  the  art  in  the  United  States 
today.    During  a  study  of  more  than  two  dozen  sites  over  a  wide  range 
of  geographic  locations,  the  ACQAP  produced  a  set  of  basic  program 
procedures,  a  scoring  method,  and  what  appears  to  be  a  sound  basis 
for  planning,  operating  and  assessing  quality  care  assurance  programs. 
The  literature  relative  to  quality  assurance  in  ambulatory  care  was 
investigated  and  a  collection  of  the  better  references  accumulated. 

Volume  I  of  the  three-volume  report  contains  a  detailed  description 
of  the  ACQAP  Model,  its  scoring  system,  and  its  findings.    The  data 
collection  instrument,  a  226-item  open-ended  questionaire  used  in 
the  study,  is  contained  in  an  appendix. 

Volume  II  contains  detailed  descriptions  of  the  27  programs  reviewed 
in  the  course  of  the  ACQAP  accompanied  by  a  compact  tabulation  of 
their  key  characteristics  for  quick  reference,  so  that  the  reader 
may  compare  the  study  sites  with  his  own. 

Volume  III  is  a  bibliography  containing  a  general  list  of  the  articles 
and  other  references  found  to  be  most  useful  to  the  study.    The  list 
is  duplicated  under  several  topic  headings  for  the  convenience  of  the 
reader  interested  in  a  particular  aspect  of  quality  assurance  in 
ambulatory  medical  care.    A  number  of  the  references  have  been 
abstracted  and  the  volume  also  contains  a  narrative  review  of  the 
literature. 
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INTRODUCTION 


This  volume  is  the  result  of  a  review  of  quality  assurance  literature 
completed  as  part  of  a  research  grant  and  project  work  funded  by  The  Robert 
Wood  Johnson  Foundation  and  the  Bureau  of  Quality  Assurance,  Department  of 
Health,  Education  and  Welfare.    The  review  was  conducted  at  various  times 
during  a  two-year  period,  culminating  in  selection  of  particular  articles 
for  abstracting.    There  are  more  than  200  general  citations  and  nearly  four 
dozen  abstracts.    The  review  concentrated  on  material  written  between  1970 
and  April,  1976,  unless  material  from  an  earlier  period  was  considered  unique 
or  essential  to  historical  perspective  and  orientation. 

In  company  with  the  first  two  volumes  of  the  report,  this  review  of 
the  literature  provides  a  concise  overview  of  the  state  of  the  art  in  ambu- 
latory medical  care  quality  assurance  today.    Compiled  primarily  for  specific 
project  uses,  it  began  to  reveal  patterns  of  various  areas  of  interest; 
hence  the  citations  and  abstracts  are  included  not  only  for  general  orienta- 
tion in  the  field  but  for  use  in  research  and  in  operational  quality  assurance 
programs.    The  indices  are  unique  in  that  works  are  classified  not  just 
according  to  topic  area,  but  also  by  setting.    "Topics"  are  subdivided  into 
fifteen  categories,  for  example,  and  eleven  different  "settings"  (e.g.,  solo 
practice,  emergency  room)  to  increase  the  utility  of  this  review  of  literature. 

There  are  five  sections  in  this  volume:  Main  Index,  Topic  Index, 
Setting  Index,  Abstracts  and  Review  of  the  Literature;  each  of  them  is 
discussed  below. 


MAIN  INDEX 

The  Main  Index  section  lists  all  articles  alphabetically  by  author. 
All  articles  are  coded  by  type,  topic  and  setting.    Letter  codes  appear  at 
the  end  of  each  article  in  the  Main  Index  section  and  in  all  the  other 
sections.    The  example  below  illustrates  a  citation  in  the  Main  Index. 


EXAMPLE 


*Brook,  Robert  H.  and  Stevenson,  Robert  L.,  Jr.:    Effectiveness  of 
patient  care  in  an  emergency  room,  New  Engl  J  Med  283:904-907, 

-  „  m  T  E£2f5.  T 

Abstract  Type         Topic  Setting 


Abstract:     An  asterisk  preceding  the  author's  name  indicates  the 

article  has  been  abstracted. 
Type       :     Type  refers  to  the  general  content  of  the  article  as 

either  Theoretical  (T)  or  Program  Description  (P-D) 

as  defined  on  page  3. 
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Topic     :     Fifteen  topics  are  defined  on  pages  3  to  5.  They 

were  selected  by  project  staff  to  identify  key  concepts 
and  concerns  in  researching,  designing,  or  describing 
quality  assurance  methods  and  programs. 

Setting  :     Nine  different  settings  defined  on  page  5  were  identified 
by  project  staff.    They  refer  to  the  specific  type  of 
medical  care  delivery  or  review  systems. 

TOPIC  INDEX 

The  Topic  Index  section  lists  articles  alphabetically  by  author  according 
to  topic.    Hence,  all  the  articles  having  the  topic  code  CA  will  appear  under 
the  topic  heading  Computer  Applications  (CA)  as  on  pages  23  and  24;  those 
having  the  topic  code  CS  will  appear  under  Consumer  Satisfaction  (CS)  as  on 
pages  25  and  26,  etc. 

SETTING  INDEX 

The  Setting  Index  section  lists  articles  alphabetically  according  to 
practice  setting.    For  instance,  articles  having  a  Setting  code  S-7,  i.e., 
articles  describing  research,  policy  or  programs  relating  to  prepaid  group 
practices,  will  appear  under  S-7  on  pages  71  and  72. 

ABSTRACTS 

Abstracts  were  made  of  44  articles  on  the  basis  of  the  following 
questions: 

1.  Is  the  article  clear,  concise  and  easily  understood? 

2.  Do  the  research  design  and  method  seem  to  be  wel 1 -constructed 
and  internally  consistent?    Was  the  method  pretested? 

3.  What  are  the  limitations  of  the  research  or  the  system 
described? 

4.  Were  the  data  collected  actually  measurable?    Was  the  sample 
representative  and  of  an  adequate  size?    Was  the  analysis 
accurate  and  thorough,  given  the  data? 

5.  Is  the  article  current  and  relevant  to  current  issues  in  the 
field? 

6.  Are  the  authors  reputable  in  the  field? 

7.  Does  the  article  add  to  the  cross-section  of  literature 
reviewed  in  Volume  III  of  this  report? 

Abstracts  are  listed  alphabetically  by  author  in  this  section. 

In  all  indexes,  abstracted  articles  are  identified  by  an  asterisk 
preceding  the  author's  name. 

REVIEW  OF  THE  LITERATURE 

Representative  articles  are  summarized  under  major  content  areas  at 
the  end  of  this  volume. 
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DEFINITIONS  AND  CODES  OF  TYPES,  TOPICS,  AND  SETTINGS 


The  definitions  are  not  precise  and  exclusive.    Many  of  the  articles 
reviewed  do  attempt  to  construct  precise  definitions.    Therefore,  types, 
topics  and  settings  are  defined  loosely  to  encompass  the  latitude  of  the 
literature  and  to  limit  their  number  to  a  manageable  size. 


DEFINITION  CODE 

TYPE 

Theoretical,  Policy  Issues  -  material  discussing  (T) 
current  aspects  of  quality  assurance  philosophy, 
general  approach,  and  construction  of  models  at 
local,  state,  and  federal  levels. 

Program  Description  -  material  discussing  the  (P-D) 
testing  or  application  of  quality  assurance 
methods  within  a  particular  setting. 

TOPIC 

Computer  Applications  -  the  use  of  computer  soft-  CA 
ware  and  hardware  in  the  processing  of  medical 
care  data. 

Consumer  Satisfaction  -  patients'  attitudes  toward  CS 
health  care  providers,  the  health  care  system  and 
the  care  received. 

Corrective  Action  -  the  various  efforts  made  to  CX 
correct  deficiencies  uncovered  by  medical  care 
evaluation.    This  category  also  includes  feedback 
and  continuing  education. 

Criteria  and  Standards  Development  -  the  process  CSD 
of  establishing  standards  by  which  medical  care 
is  evaluated. 

Data  Collection,  Processing,  Analysis  -  the  DC 

various  ways  data  are  collected,  processed  and 

analyzed.    Articles  dealing  with  the  concerns 

of  confidentiality  and  privacy  are  included 

here. 
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DEFINITION 


TOPIC 

Evaluation  and  Impact  Studies  -  analysis  of 
medical  care,  comparing  the  care  to  what  is 
possible,  and  attempting  to  determine  the 
origins  of  deviations  while  stressing  improve- 
ment.   This  is  a  general  category;  articles 
listed  herein  may  or  may  not  address  process, 
outcome  or  structural  evaluation. 

Outcome  Evaluation  -  analysis  of  medical  care 
which  examines  end  result  of  care  in  terms  of 
patient  mortality,  physical  or  psychological 
morbidity,  care,  rehabilitation,  level  of 
function,  etc. 

Process  Evaluation  -  analysis  of  what  is  done 
to  the  patient  (history  taking,  lab  tests, 
diagnosis,  therapy,  etc.)  as  compared  to  a 
standard . 

Structural  Evaluation  -  analysis  of  the  effect 
that  characteristics  of  facilities  and  providers 
(e.g.,  hospital  beds,  physicians  per  population, 
board  certification,  etc.)  have  on  medical  care. 
This  category  also  includes  accreditation  and 
the  use  of  paraprofessionals. 

Medical  Audit  -  the  process  which  measures  actual 
medical  care  received  compared  to  a  set  of 
standards. 

Medical  Record  -  the  primary  form  of  patient  data 
collection.    Protocols  are  also  included  in  this 
category. 

Peer  Review  -  the  evaluation  of  a  physician  by 
physicians,  usually  of  the  same  area  and  specialty 

Provider-Patient  Relationship  -  the  quality  of 
communication  between  providers  and  consumers 
of  medical  care.    Consumer  involvement  in  decision 
making  is  also  included  in  this  category. 

Quality  (Assessment,  Assurance)  -  articles  in  this 
category  attempt  to  make  explicit  what  is  meant  by 
the  quality  of  medical  care. 


DEFINITION  CODE 

TOPIC 

Utilization  Review  -  the  process  which  measures  UR 

the  use  of  available  facilities  and  services  to 
determine  medical  necessity,  cost  effectiveness, 
and  conformity  to  criteria  for  optimal  use. 

SETTING 

Group  Practice,  Fee  for  Service  S-l 

Hospital,  Emergency  Room  S-2 

Hospital,  Inpatient  S-3 

Hospital,  Outpatient  Clinic  S-4 

Medical  Care  Foundation  S-5 

Mental  Health  Center  S-6 

Prepaid  Group  Practice  S-7 

HMO  or  NHC  types  S-7a 

Kaiser  Types  S-7b 

PSRO  S-8 

Solo  Practice  S-9 
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Blum,  Henrik,  L.:    Evaluating  health  care,  Med  Care  12:999-1011 ,  Dec  1974 
(T),  ES,  ES-0,  ES-P,  ES-S,  MA,  UR 

PSRO  legislation  is  seen  to  have  led  to  a  focus  on  process  types 
evaluation  and  thus  to  cost-control  approaches  to  improvement.  The 
author  considers  quality  as  embracing:    1)  patient's  health  status 
and  health  attitudes  on  entry  to  care,  2)  suitability  of  the  care 
delivery  machinery  (structure),  3)  application  of  care  (process), 
and  4)  outcomes  of  current  application  of  care. 

Seven  evaluatory  approaches  are  discussed,  though  the  author  emphasizes 
the  "Past-Focused  Analysis."    1)  The  traditional  answer  of  "more 
and  better  of  everything"  that  relates  to  health  (i.e.,  do  more  research, 
require  higher  levels  of  training)  is  seen  as  an  anti-evaluation 
approach.    2)  The  evaluation  of  the  community  level  of  well-being 
as  a  base  from  which  to  seek  improvement  is  limited  because  the  ultimate 
outcome  of  how  well  people  are  is  only  very  modestly  the  result  of 
their  health  care.    3)  In  the  Prospective  Outcome  Analysis  as  a  means 
of  identifying  needed  improvements,  an  expert  committee  establishes 
the  standards  of  specified  outcomes  for  selected  conditions.  Statistically 
significant  numbers  of  cases  are  checked  and  when  outcomes  are  signi- 
ficantly lower  than  expectations  the  committee  investigates  for  the 
underlying  factors  contributing  to  the  failure.    The  committee  then 
designs  and  tries  to  institute  changes  required  to  build  further 
failures  out.    However  all  failures  cannot  be  attributed  to  the  health 
care  process  because  the  many  environmental  factors  impinging  upon 
the  patient  can  have  much  to  do  with  poor  outcome.    4)  The  Past-Focused 
Analysis  is  predominantly  an  effort  of  looking  back  into  what  the 
patient  brings  with  him,  including  prior  health  care,  to  find  the 
causes  that  bring  a  patient  to  care  in  the  condition  in  which  he 
is  found.    Forces  identified  as  preventable  immediately  can  lead 
to  spelling  out  measures  that  would  eliminate  those  forces  as  the 
causes  of  the  particular  condition.    5)  Using  Process  Evaluation 
to  find  failures  in  the  health  care  machinery  does  not  necessarily 
give  a  real  idea  of  what  is  going  wrong  or  why,  nor  does  an  apparently 
ideal  process  guarantee  good  care.    The  patient's  environmental  situ- 
ation may  justifiably  require  departures  from  the  ideal.  However, 
Process  Evaluation  should  involve  physicians  and  other  health  care 
workers  in  continuing  education  and  practice  improvement.    6)  In 
Structural  Evaluation  the  expert  committee  works  backward  from  their 
ideal  process  to  the  ideal  structure  to  implement  those  processes. 
However,  since  structure  is  not  specifically  checked  against  outcomes, 
it  can  promote  costly  changes  which  may  not  be  improvements  at  all. 
7)  A  "tracer"  is  a  condition  that  has  a  significant  functional  impact, 
can  be  diagnosed,  occurs  frequently,  requires  much  care,  has  effective 
therapy,  whose  epidemiology  is  comprehended,  is  common  in  a  specific 
age-sex-race  grouping  and  uses  many  major  kinds  of  health  services. 
The  Tracer  method  offers  the  possibility  of  examining  each  major 
aspect  of  quality  control. 
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The  author  concludes  in  this  general  summary  and  review  of  each  type 
of  quality  evaluation  that  not  all  investment  should  go  into  one 
or  two  types  of  evaluation  alone.    The  process  evaluation  ignores 
the  patient's  contributions  to  the  outcome  of  the  processes,  but 
the  past-focused  and  the  prospective  outcome  analyses  point  out  the 
process  and  structural  failures.    The  use  of  well-studied  tracer 
conditions  may  become  a  particularly  efficient  and  effective  way 
of  relating  all  types  of  quality  evaluation  of  health  care. 
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Bouchard,  R.E.,  Eddy,  W.M.,  Tufo,  H.M.,  Twitchell,  J.C.,  Van  Buren,  H.C., 
and  Bedard,  L.:    The  patient  and  his  POMR,  Quality  Assurance  of  Medical 
Care,  DHEW  (HSM)  73-7021,  Feb  1973,  pp.  233-239  (P-D),  CSD,  CX,  ES-P, 
M7~MR,  PPR,  UR 

A  model  ambulatory  care  center  (the  Given  Health  Care  Center)  was 
organized  at  the  University  of  Vermont  according  to  the  Weed  Problem- 
Oriented  System  which  is  based  on  the  four  phases  of  medical  action: 
1)  collecting  the  data  base;  2)  formulating  problems;  3)  making  explicit 
plans;  and  4)  conducting  a  follow-up  based  on  specific  goals.  After 
freshman  medical  students,  who  are  actually  laymen,  proved  to  be 
excellent  auditors,  it  was  decided  to  send  a  copy  of  the  record  to 
all  patients.    The  goals  were  to  educate  the  patient  and  define  his 
responsibility  to  carry  out  the  health  care  plan.  Provider-patient 
communication  is  not  good  after  a  complete  examination  and  some 
long-range  plans  are  not  clearly  established  while  the  patient  is 
still  in  the  office.    The  patients  could  also  participate  in  the 
audit  process.    Improved  understanding  and  motivation  are  potential 
advantages,  but  possible  disadvantages  include  increased  anxiety 
level  and  an  information  overload. 

After  6  months  experience,  a  questionnaire  was  mailed  to  100  patients 
receiving  complete  examinations  and  there  was  a  93%  response  rate. 
For  each  of  a  variety  of  health  factors,  the  patients  were  clearly 
in  favor  of  getting  more  detailed  information.    Respondents  also  indi- 
cated an  increased  understanding  of  their  condition  and  role  in 
improving  it.    Eighty  percent  were  now  more  careful  in  taking  medication 
and  78%  indicated  that  they  had  subsequently  changed  their  living, 
eating  or  drinking  patterns.    Ninety-seven  percent  felt  that  after 
reviewing  their  own  record  they  were  less  worried  about  their  health. 

A  process  audit  method  has  been  chosen  and  is  a  positive  experience 
because  it  leads  to  continuing  self-education,  motivation,  uniformity 
of  performance,  and  updating  standards.    The  patient  participates  in 
three  sections  of  the  audit:    1)  the  role  and  interaction  description 
of  the  patient;  2)  subjective  data;  and  3)  clarity  of  the  plans. 
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Brook,  Robert  H.:    Quality  assurance:    the  state  of  the  art,  Hosp  Med 
Staff  15-26,  Mar  1974  (T),  CSD,  ES-0,  ES-P 

Although  the  United  States  is  unique  in  its  commitment  to  health 
care  quality  assurance  at  the  level  of  the  individual  practitioner, 
Brook  does  not  see  this  as  necessarily  resulting,  even  if  implemented 
with  the  most  unselfish  of  motives,  in  improvement  in  the  health 
of  the  population  or  in  stabilization  or  reduction  in  the  costs  of 
health  services.    It  is  just  as  likely  that  it  would  result  in  little 
or  no  improvement  in  the  general  health  at  a  much  greater  cost. 
He  feels,  however,  that  the  causal  relationship  between  a  quality 
assurance  program  and  changes  in  the  health  of  the  population  and 
in  the  costs  of  the  health  services  can  and  should  be  tested. 

Because  the  maintenance  or  improvement  of  patient  outcome  is  the 
goal  of  the  medical  care  system,  outcome  or  clinical  and  structural 
factors  shown  to  be  related  to  it  should  be  the  basis  of  quality 
assessment.    This  is  made  difficult,  however,  because  1)  medical 
care  services  are  not  the  only  factors  affecting  outcome,  2)  measuring 
outcome  is  difficult  and  expensive,  and  3)  the  outcome  of  care  varies 
as  a  function  of  the  stage  of  the  disease.    For  these  reasons  the 
trend  is  to  examine  and  develop  process  and  structure  criteria  as 
yardsticks  for  assessing  the  quality  of  health  care.    However,  the 
causal  relationships  between  both  process  and  structure  criteria 
and  outcome  are  not  based  on  hard  scientific  evidence  and  this  leads 
to  guidelines  that  if  followed  would  not  result  in  improvement  in 
the  health  of  the  American  people  but  would  result  in  a  vast  increase 
in  the  cost  of  medical  care.    Studies  indicate  that  process  criteria 
selected  by  practicing  physicians  tend  to  be  unrelated  to  outcome 
and  that  these  physicians  tend  to  ignore  their  own  criteria. 

Improvements  in  the  health  of  the  population  will  not  come  as  a  result 
of  following  process  criteria,  nor  having  physicians  retake  board 
examinations  and  attend  continuing  education  courses.    The  best  research 
techniques  must  be  used  to  find  what  works  and  what  does  not.  Change 
will  be  necessary  in  both  institution  and  provider  behavior. 
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Brook,  Robert  H.  and  Appel,  Francis  A.:    Qua! ity-of-care  assessment: 
choosing  a  method  for  peer  review,  New  Engl  J  Med  288:1323-1329, 
21  June  1973  (P-D),  ES-P,  ES-0,  CSD. 

A  successful  peer  review  system  depends  upon  whether  process  or 
outcome,  or  both,  data  are  used  and  upon  by  whom  and  how  value  judgements 
are  made  to  determine  the  quality  of    ire.    To  answer  these  questions, 
comparisons  were  made  of  5  different  methods  of  quality  assessment: 
1)  implicit  (relying  on  the  subjective  opinion  of  an  individual 
judge)  judgement  of  process;  2)  implicit  judgement  of  outcome;  3) 
implicit  judgement  of  process  and  outcome  combined  (qual ity-of-care 
judgement);  4)  explicit  (relying  on  predetermined  criteria  set  by 
group  agreement)  judgement  of  process;  and  5)  explicit  judgement 
of  outcome. 

Three  conditions  (urinary  tract  infection,  hypertension,  and  ulcer- 
ated lesion  in  the  stomach  or  duodenum)  were  selected  for  this  study 
conducted  at  the  Baltimore  City  Hospitals.    For  the  three  implicit 
methods  physician  judges  read  an  abstract  prepared  from  the  medical 
record  and  patient  interview.    The  abstract  was  more  legible,  more 
complete,  and  more  in  chronologic  order  than  the  medical  record. 
Each  of  three  physicians  performed  implicit-process  judgement  from 
data  provided  on  the  first  page  of  the  abstract,  containing  background 
information  and  medical-process  data.    Then  the  second  page  of  the 
abstract  (outcome  data)  was  read  and  the  physicians  decided  whether 
the  outcome  could  have  been  improved  if  the  medical -care  process 
had  been  better  (implicit-outcome  judgement).    The  physicians  then 
rated  the  quality  of  care  received  by  the  patient  as  acceptable 
or  unacceptable  (implicit  qual ity-of-care  judgement).  Criteria 
for  each  of  the  three  conditions  necessary  to  provide  good  care 
and  likely  to  affect  outcome  in  an  important  way,  were  those  selected 
by  five  of  seven  physicians,  and  when  applied  to  each  case,  resulted 
in  an  explicit-process  judgement.    For  groups  of  patients  with  similar 
characteristics  likely  to  affect  prognosis  physicians  estimated 
outcome  with  no  therapy,  with  current  therapy,  and  with  adequate 
therapy  (explicit-outcome  judgement). 

At  the  end  of  the  5-month  study,  44%  of  the  hypertensive  patients 
still  alive  had  uncontrolled  blood  pressure,  probably  due  to  insuf- 
ficient follow-up,  noncompliance  with  medication,  and  inadequate 
adjustment  of  drug  dosage.    Sixty-one  percent  of  patients  with  urinary 
tract  infection  had  bacteria  resistant  to  the  antibiotic  originally 
chosen.    Sixty-one  percent  of  patients  with  an  ulcerated  lesion 
remained  symptomatic. 

With  at  least  two  of  three  judges  considering  care  adeqate,  23.3% 
patients  had  an  adequate  medical -care  process.    Outcome  was  judged 
unimprovable  in  63.2%  of  the  cases  and  for  27.1%  of  the  cases  the 
qual ity-of-care  was  judged  acceptable.    Only  2%  of  the  cases  met 
all  the  explicit-process  criteria  and  the  mean  percentages  of  criteria 
met  were  52%  (urinary  tract  infection),  58%  (hypertension),  and 
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35%  (ulcerated  lesion).    Explicit-outcome  evaluation  resulted  in 
40%  of  urinary  tract  infection  patients  and  44%  of  hypertensive 
patients  receiving  acceptable  care.    For  those  with  ulcerated  lesions 
outcome  was  found  to  be  worse  than  the  estimated  outcomes  with  no 
therapy. 

Implicit-process  judgement  correlated  weakly  with  outcome.  The 
three  implicit  methods  varied  significantly  with  the  conditions 
and  reliability  was  not  sufficient  to  judge  individual  patient  care. 
In  terms  of  physician  willingness  to  estimate  and  the  validity  of 
the  estimates,  explicit-outcome  evaluation  seemed  the  least  satis- 
factory.   Explicit-process  evaluation,  currently  the  most  widely 
used,  produced  the  severest  judgements. 
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Brook,  Robert  H.  and  Stevenson,  Robert  L.,  Jr.:    Effectiveness  of  patient 
care  in  an  emergency  room,  New  Engl  J  Med  283:904-907,  22  Oct  1970 
(P-D),  ES-0,  ES-P 

To  determine  if  the  emergency  room  is  a  suitable  source  of  primary 

care  for  inner-city  residents,  an  evaluation  of  diagnostic  and  therapeutic 

processes  and  outcomes  was  conducted  for  a  sample  of  141  emergency 

room  patients  scheduled  for  upper  gastrointestinal  series,  barium 

enemas,  or  oral  cholecystography.    The  study  was  administered  during 

the  months  of  April-June,  1969,  at  Baltimore  City  Hospital.  The 

Williamson  model  was  used. 

Patients  for  study  were  obtained  from  x-ray  department  lists  by  date 

and  source  of  appointment.    A  retrospective  review  of  each  chart 

and  all  x-ray  reports  obtained  from  the  x-ray  department  was  done 

in  addition  to  a  patient  interview.    Results  are  presented  in  four 

parts:    the  success  of  a  follow-up  study,  patient  characteristics, 

diagnostic  processes  and  outcomes,  and  therapeutic  processes  and 

outcomes.    Patient  data  were  analyzed  according  to  employment,  sex, 

race  and  source  of  payment.    Some  of  the  results  were  as  follows: 

Out  of  141  patients,  94  (67%)  completed  diagnostic  x-ray  studies; 

77  (55%)  received  an  adequate  work-up  according  to  established  criteria; 

of  38  patients  with  abnormal  x-rays,  37%  had  no  adequate  therapy. 

Conclusions:    Effective  medical  care  was  rendered  for  only  35  (27%) 
of  the  patients,  ineffective  care  for  84  (60%)  of  the  patients,  and 
19  patients  experienced  no  net  positive  or  negative  effect.    By  every 
criterion  included  in  the  study,  the  medical  care  in  the  emergency 
room  appeared  to  be  both  inefficient  and  inadequate.  Approximately 
one-third  of  the  patients  were  forced  to  seek  additional  help,  had 
prolonged  symptoms  and  missed  work.    Effective  medical  care  was  delivered 
to  only  one-fourth  of  the  patients. 
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Brown,  Clement  R.  and  Uhl,  Henry  S.N.:    Mandatory  continuing  education 
sense  or  nonsense?    JAMA  213:1660-1667,  7  Sept  1970  (P-D),  CX 

Although  research  in  medical  education  continues,  implementation 
of  experimental  programs  has  not  been  successful.    The  history  of 
continuing  medical  education  has  led  us  to  the  standard  teacher  or 
planner-oriented  approach.    This  method  conceptually  does  not  relate 
educational  programs  directly  to  identified  physician  and  patient 
need.    This  can  be  accomplished  by  1)  identification  of  deficiencies 
and  combining  them  with  corrective  programs  and  2)  establishment 
of  a  communication  network  to  provide  regular  consultation. 

The  "bi-cycle"  concept  directly  relates  patient  care  to  education. 
Both  the  patient  care  and  educational  cycles  begin  with  the  physician- 
patient  interaction.    Education  takes  place  where  such  interaction 
occurs.    Data  from  these  interactions  is  abstracted  and  processed. 
Management  criteria  (screening,  prevention,  diagnosis,  and  therapy) 
are  established  for  disease  categories  causing  the  most  amount  of 
preventable  disability.    Actual  physician  performance  is  compared 
to  the  criteria  and  gaps  are  identified,  representing  improvement 
potential.    Closing  these  gaps  is  the  educational  objective.  Here 
the  bi-cycle  concept  enters  into  the  inner  educational  cycle,  where 
causes  (cognitive,  skill,  or  attitudinal)  of  the  gaps  are  determined. 
Effectiveness  of  the  proposed  educational  programs  is  determined 
by  reentry  into  the  outer,  patient-care  and  audit  cycle. 

The  bi-cycle  concept  was  employed  to  study  the  use  of  antibiotics 
at  the  Chestnut  Hill  Hospital.    The  medical  staff  estimated  a  minimal 
expectation  of  75%  appropriate  antibiotic  use  and  a  35%  audit  result. 
An  audit  by  chart  review  found  30%  appropriate  use  and  deemed  it 
unacceptable.    Six  of  the  first  eight  patients  receiving  appropriate 
antibiotic  therapy  was  the  educational  objective.    An  examination 
using  some  of  the  originally  audited  cases  revealed  an  average  of 
68%  appropriate  choices.    After  four  conferences  on  antibiotic  use 
using  the  examination  as  the  focus,  physician  performance  was  reaudited 
(back  to  the  outer  patient-care  cycle)  and  found  to  have  improved 
to  48%  and  subsequently  to  60%.    A  required  progress  rate  explaining 
the  necessity  of  prescribed  antibiotics  was  another  instituted  change. 

Two-way  closed  circuit  television  for  consultative  purposes  is  feasible 
and  provides  a  regular  highly-motivated  problem-solving  learning 
process.    A  WATS  system  has  proved  feasible  and  worthwhile  in  the 
Alabama  Regional  Medical  Program  and  such  has  been  the  experience 
of  the  Dartmouth  Medical  Center  with  its  television  program.  It 
is  hoped  that  basing  accreditation  on  standard  types  of  continuing 
education  will  not  rule  out  experimental  programs  which  are  more 
effective. 
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Buck,  Charles  R.,  and  White,  Kerr  L.:    Peer  review:    impact  of  a  system 
based  on  billing  claims,  New  Engl  J  Med  291:877-883,  24  Oct  1974 
(P-D),  PR 

To  determine  if  adjustment  of  billing  claims  made  by  a  peer  review 
mechanism  affects  subsequent  practice  patterns,  the  records  of  Medi- 
Cal  patients  of  the  San  Joaquin  Foundation  for  Medical  Care  were 
examined  for  the  period  from  1  February  1968  to  31  May  1971.  The 
Foundation  has  performed  billing  claims  peer  review  since  1955  to 
control  unnecessary  physician  services  and  assure  adequate  funds 
to  pay  for  required  services.    All  claims  are  reviewed  according 
to  the  Foundation's  standards  by  nonphysician  examiners  but  may  be 
then  referred  to  a  physician  or  the  Medical  Review  Committee  for 
adjustment. 

Fifteen  procedures  in  three  categories  (injections,  physician  visits, 
and  laboratory  procedures),  performed  by  259  individually  identified 
physicians,  were  studied.    The  uneven  distribution  of  adjusted  claims 
that  was  revealed  tends  to  corroborate  the  impression  that  such  review 
mechanisms  focus  more  on  some  providers  than  on  others.  The  study 
placed  major  emphasis  on  attempting  to  associate  individual  effects 
with  peer  review  stimuli.    To  measure  the  impact  of  peer  review, 
changes  in  the  trend  (the  rate  and  direction  of  change  in  the  number 
of  procedures  provided  per  month  during  both  a  "before"  and  an  "after" 
period)  were  illustrated  by  calculating  and  comparing  the  slopes 
of  the  regression  lines  for  each  physician  for  each  specific  procedure 
during  both  periods.    A  statistical  analysis  was  done  to  test  for 
an  association  between  the  percentage  of  claims  adjusted  for  a  specific 
procedure  and  the  magnitude  of  any  "before"  to  "after"  change  in 
regression  line  slopes. 

A  positive  association  was  seen  between  the  percentage  of  procedures 
adjusted  and  the  subsequent  decreases  in  regression  line  slopes  although 
the  tests  for  an  overall  decrease  in  regression  line  slopes  were 
statistically  significant  for  only  four  of  the  15  procedures  studied. 
Physicians  with  review  adjustments  to  more  than  20  per  cent  of  their 
claimed  injections  decreased  their  regression  line  slopes  more  than 
the  remaining  physicians  did. 

Due  to  the  long  history  of  claims  peer  review  at  the  Foundation, 
much  of  the  reduction  in  variation  among  practice  patterns  occurred 
before  periods  concerned  in  this  study.    There  was  inconsistency 
in  the  results  suggesting  that  for  some  physicians  or  procedures 
the  stimuli  may  have  a  decreasing  effect  and  that  in  some  cases  the 
same  level  of  change  might  be  achieved  with  lower  percentage  levels 
of  adjustments. 
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Curran,  William  J.,  Stearns,  Barbara  and  Kaplan,  Honora:    Privacy,  confiden- 
tiality and  other  legal  considerations  in  the  establishment  of  a 
centralized  health-data  system,  New  Engl  J  Med  281:241-248,  31  July 
1969  (T),  DC 

The  protection  of  privacy  and  confidentiality  are  the  most  important 
legal  questions  in  the  development  and  acceptance  of  a  centralized 
health-data  system.    The  right  of  privacy  is  the  right  to  keep  informa- 
tion about  one's  self  inaccessible  to  others.    Confidentiality  implies 
disclosure  of  certain  information  to  others  for  particular  purposes 
and  with  certain  restrictions.    Testimonial  privilege  means  information 
received  confidentially  cannot  be  divulged  in  a  legal  proceeding 
without  consent. 

In  Massachusetts  the  law  states  that  all  records  in  the  custody  of 
state,  county  or  local  governmental  agencies  are  public  records  with 
certain  exceptions,  including  personal  and  medical  information. 
Hospital  patient  records,  owned  by  the  hospitals,  are  available  to 
the  individual  patients  except  those  under  the  control  of  the  Depart- 
ment of  Mental  Health.    Unlike  most  states,  Massachusetts  does  not 
provide  for  physician-patient  testimonial  privilege.  Professional 
group  ethical  codes  forbidding  disclosure  of  patient  confidences 
are  legally  binding.    The  release  of  aggregate  data  where  individuals 
cannot  be  identified  does  not  seem  to  pose  a  problem. 

The  Public  Information  Act  of  1966  made  federal  records  more  accessible 
to  the  public,  except  for  personnel  and  medical  data.    Census  data, 
unidentifiable  as  to  individuals  or  establishments,  is  publicly  available. 
The  Social  Security  Administration  protects  identifiable  data  and 
names  of  contributors  from  disclosure. 

Only  a  general  outline  of  the  legal  protection  of  the  "right  to  privacy" 
has  been  developed.    Safeguards  must  be  developed  to  protect  data 
aggregately  released  but  identifiably  stored.    A  centralized  health- 
data  system  should  develop  rules  and  regulations  specifying  conditions 
for  information  release  and  should  periodically  re-evaluate  them 
through  a  privacy  and  confidentiality  committee.    There  should  be 
criminal  penalties  for  violations  and  civil  damages  should  be  awarded 
for  injuries  resulting  from  such  violations.    Violation  of  the  system's 
rules  and  regulations  by  employees,  associates  and  users  should  result 
in  dismissal  and  termination  of  association.    It  is  recommended  that 
the  major  contributors  and  users  of  the  data  make  up  a  large  part 
of  the  system's  policy-making  body. 
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This  article  describes  a  system  of  ongoing  evaluation,  using  the 
problem-oriented  audit,  to  successfully  incorporate  continual  feedback 
of  relevant  information  to  the  clinical  staff  for  their  education 
and  for  the  improvement  of  medical  care  delivery.    The  study  took 
place  at  the  Whitney  M.  Young,  Jr.,  Community  Health  Center  which 
provides  family-oriented  health  care  to  a  low-income  inner  city  popula- 
tion. The  project  was  in  part  federally  funded  and  began  in  July, 
1971.    Four  health  problems  (pediatric  otitis  media,  adult  hypertension, 
adult  urinary  tract  infection,  and  pediatric  anemia)  were  selected 
as  tracers  to  assess  the  quality  of  health  care.    The  health  center 
physicians  designed  protocols,  including  criteria  for  resolution 
or  control,  for  diagnostic  evaluation  and  treatment  of  these  four 
health  problems.    Audit  forms  were  designed  as  well  as  a  system  of 
computerized  storage  of  selected  clinical  and  statistical  data. 
Specific  items  were  selected  to  assess  provider  compliance  with  the 
protocols,  and  data  were  collected  concerning  patient  compliance 
with  therapeutic  regimens  and  scheduled  appointments. 

The  study  found  that  failure  to  record  data,  misunderstanding  or 
disagreement  concerning  what  data  are  necessary,  and  frequent  failure 
of  patients  to  keep  return  appointments  were  all  of  major  importance 
and  operative  to  some  extent  in  the  majority  of  clinical  episodes. 
The  audits  brought  out  the  diversity  of  the  clinical  staff's  opinions 
and  understanding  regarding  criteria  for  diagnosis,  and  for  either 
resolution  or  adequate  control  of  the  various  clinical  entities. 
After  completion  of  the  audit  the  disparities  between  the  protocol 
and  actual  practice  were  defined  and  presented  to  the  clinical  staff 
for  an  attempt  toward  closer  adherence  to  or  a  modification  of  the 
protocols.    A  limitation  of  the  audit  methods  is  that  it  is  dependent 
upon  the  completeness  of  the  recording  of  clinical  data.    The  study 
also  concluded  that  the  selection  of  criteria  for  health  care  evaluation 
should  actively  involve  all  clinical  providers. 
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The  processes  of  care  for  appendicitis  and  myocardial  infarction 
were  analyzed  to  assess  the  relationship  between  the  recorded  process 
of  medical  care  and  the  outcome  of  that  care.    A  medical -record 
librarian  abstracted  information  from  patients'  records  with  only 
2.4%  error.    A  list  of  recorded  symptoms,  signs  and  laboratory  data 
relevant  to  appendicitis  and  derived  from  50  charts  from  one  study 
hospital  was  used  to  audit  all  150  charts  from  three  hospitals. 
Another  list,  containing  44  elements  of  history,  physical  examination, 
and  various  tests,  and  documented  on  at  least  75%  of  50  charts  from 
one  hospital,  was  created  for  myocardial  infarction.    The  corresponding 
outpatient  charts  were  reviewed  for  the  following  outcomes:    time  before 
returning  to  work;  angina  pectoris;  congestive  heart  failure  or 
arrhythmias  not  recorded  on  medical  record;  new  myocardial  infarction; 
and  death.    These  charts  were  also  examined  for  items  found  to  be 
related  to  myocardial  infarction  outcomes:    white-cell  count,  blood 
pressure,  congestive  heart  failure,  diabetes,  and  histories  of 
myocardial  infarction  and  hypertension.    Three  cardiologists  also 
drew  up  an  audit  list  for  myocardial  infarction. 

Differences  among  the  three  hospitals  in  their  recording  of  useful 
information  concerning  appendicitis  were  statistically  significant. 
Yet  they  correctly  diagnosed  the  condition  a  similar  percentage  of 
times  (outcome  of  the  clinical  diagnosis  was  the  same).  Similarly, 
for  myocardial  infarction  there  was  no  correlation  between  recorded 
data  and  the  outcomes  after  discharge.    Thus,  chart  review  of  the 
process  of  care,  the  usual  method  of  audit,  may  be  misleading. 
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A  preliminary  review  of  the  Illinois  Hospital  Admission  and  Surveillance 
Program  (HASP),  a  peer  review  program,  was  conducted  in  1972.  Signi- 
ficant cost  reductions  and  improvement  in  both  utilization  and  quality 
of  medical  care  for  recipients  of  medical  assistance  programs  are 
goals  that  HASP  has  in  common  with  established  PSRO  principles. 
HASP  hopes  to  accomplish  these  goals  by  the  monitoring  and  precertifi- 
cation  of  hospital  admissions,  elimination  of  hospital  days  medically 
unnecessary,  and  ongoing  surveillance  of  diagnoses,  services,  and 
patterns  of  care.    A  specially  trained  coordinator  provides  the  admini- 
strative services  for  precertifi cation  of  length  of  stay,  extensions, 
summary  abstracts  and  data  collection.    To  avoid  delays  in  implementation, 
median  length  of  stays  were  taken  from  the  pooled  statistics  of  the 
Professional  Activities  Study,  representative  of  Illinois  Hospital. 
The  involvement  of  a  government  contract  officer,  enpowered  to  change 
the  original  contract  to  solve  problems,  will  also  facilitate  implementation. 

Historical  utilization  and  cost  data  is  necessary  as  well  as  a  management 
reporting  system,  based  on  the  individual  patient  abstract,  to  monitor 
expenditures,  problem  areas,  trends,  etc.    HASP  designed  both  year- 
to-date  summary  reports  and  program  exception  reports.  Physician 
advisors  were  reimbursed  for  their  time  and  responsibility.  Forms 
should  be  designed  for  minimum  paperwork  for  the  physician  and  the 
hospitals'  financial  office  should  control  certification  of  hospital 
bills  with  a  special  stamp,  not  additional  forms.    The  state  HASP 
committee,  or  its  PSRO  equivalent,  must  remain  prepared  to  interpret 
contract  policy  and  the  local  HASP  committee  should  act  as  a  court 
of  first  appeal . 

The  following  are  operational  problems  that  may  arise  and  ways  in 
which  HASP  has  addressed  these  problems.    Without  review  by  the  physician 
advisor,  the  coordinator  may  grant  extension  in  cases  of  obvious 
medical  necessity.    Because  extensions  should  be  granted  only  when 
medically  necessary,  extensions  due  to  hospital  delays  in  providing 
services  should  be  examined  carefully.    Extensions  due  to  delays 
in  long-term  care  transfer  are  granted  only  when  they  are  the  result 
of  governmental  agency  inaction  or  justifiable  social  service  problem. 
HASP  requires  a  minimum  of  four  hours  of  hospitalization  prior  to 
certification,  allows  only  one  certified  day  for  multiple  admissions 
for  false  labor,  and  usually  precertifies  one  day  for  preoperative 
care.    Diagnostic  changes  may  also  require  extensions.    Only  attending 
physicians,  and  in  teaching  hospitals  only  senior  residents,  may 
request  admission  certification.    The  physician  advisor  must  review 
admissions  by  those  doctors  who  are  continually  in  noncompliance. 
Only  after  eligibility  is  determined  by  the  appropriate  state  agency 
may  certification  occur.    Determination  of  eligibility  may  occur 
after  discharge,  in  which  case  retroactive  certification  may  be  granted. 
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PSRO's  must  not  be  too  small  and  certification  should  be  implemented 
initially  in  institutions  with  the  greatest  medical  assistance  admission 
rates  to  insure  a  high  volume  cash  flow.    About  80  patients  per  coordi- 
nator would  be  considered  a  full  work  load.    Modification  of  certification 
standards  should  happen  only  after  100,000  admissions.    A  price  tag 
for  the  one-time  start  up  costs  should  be  agreed  upon  right  at  the 
beginning  and  further  reimbursement  should  be  on  a  risk-incentive 
basis. 

The  average  length  of  stay  dropped  from  7.2  days  in  the  year  prior 
to  HASP  implementation  to  6.2  days  in  the  corresponding  period  of 
HASP  operation.    A  $9.6  million  reduction  in  hospital  expenses  was 
anticipated  for  this  period.    Although  the  number  of  eligible  recipients 
was  21%  higher  during  HASP  operation,  state  payment  for  hospital 
services  increased  only  1.5%.    HASP  expenses  accounted  for  2%  of 
total  hospital  expenditures.    PSRO's  will  eventually  expand  to  the 
private  sector  and  ambulatory  and  long-term  care  settings. 
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Process  and  structure  evaluation  is  legitimate  only  when  closely 
associated  with  outcome  measurements.    To  overcome  the  inappropriate- 
ness  and  insensitivity  of  previous  outcome  measures,  the  Sickness 
Impact  Profile  (SIP),  a  behaviorally  based  measure  of  sickness-related 
dysfunction,  an  indicator  of  health  status,  has  been  developed.  The 
provider's  definition  of  the  illness,  the  care  process,  and  the 
individual's  own  sickness  perception  influence  the  impact  of  the 
sickness.    The  behavioral  aspect  of  sickness  is  an  appropriate  outcome 
measure  because:    1)  the  overall  impact  of  illness  is  reflected  in 
behavior;  2)  both  health  providers  and  consumers  perceive  the  social, 
mental,  and  physical  effects  of  sickness;  and  3)  many  health  care 
systems  are  responsible  for  the  health  of  heterogeneous  groups  over 
long  periods  of  time. 

Patients,  health  care  professionals,  individuals  caring  for  patients, 
and  the  apparently  healthy  contributed  statements  describing  behavioral 
dysfunction.    Grouping  techniques  cut  the  original  1250  statements  to 
312,  which  were  then  sorted  into  14  categories  of  items  (e.g.  social 
interaction,  locomotion,  etc.).    The  SIP  items  were  rated  by  25  judges 
on  an  11 -point  scale  from  "minimally  dysfunctional"  to  "severely 
dysfunctional."    Then  they  rated  the  least  and  most  dysfunctional 
items  to  provide  scaled  endpoints  from  which  to  rate  the  remaining 
items.    There  was  high  agreement  between  the  judges  in  both  steps. 

In  a  pilot  study  of  the  SIP  246  group  practice  enrollees  (inpatients, 
home  care  patients,  walk-in  clinic  patients,  outpatients,  and  non- 
patients)  were  interviewed  and  their  set  of  positive  responses  (dys- 
functions) provided  a  profile  of  sickness  impacts,  a  protocol.  Four 
groups  of  25  judges  performed  a  protocol  scaling  similar  to  the  item 
scaling.    Agreement  among  judges  was  again  high  and  differences  in 
medical  sophistication  did  not  produce  significantly  different  scores. 
The  profile  scoring  method  (items  checked  in  four  scale-point  groupings 
and  the  per  cent  scoring  method  (percentage  of  total  possible  dysfuncti 
were  used.    Preliminary  tests  showed  evidence  of  reliability  and 
validity.    SIP  contains  no  considerations  relating  to  prognosis  but 
was  found  to  be  a  complete  list  of  dysfunctions. 
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To  overcome  the  shortcomings  of  structural  and  process  assessment, 
a  "staging"  concept  has  been  developed  whose  basic  premise  is  that 
"the  seriousness  of  a  patient's  condition  at  some  point  in  the  treatment 
process  is  a  good  indication  of  the  outcome  of  the  previous  parts 
of  the  process."    A  medical  problem  or  disease  is  divided  into  three 
stages:    1)  without  complications  or  with  complications  of  minimal 
severity,  2)  with  local  or  moderately  severe  complications,  and  3) 
with  systemic  or  serious  complications.    Sublevels  can  also  be  defined 
for  more  precision.    In  this  way  groups  of  patients  requiring  common 
treatment  procedures  and  services  and  having  similar  expected  outcomes 
can  be  identified.    These  "staging"  classifications  can  be  translated 
into  ICDA  and  H-ICDA  codes  for  use  by  nonphysician  personnel  and, 
in  some  cases,  directly  by  computer. 

It  is  assumed  that  an  inadequately  treated  population  can  be  expected 
on  the  average  to  be  further  advanced  in  illness  (at  a  higher-numbered 
stage)  than  an  adequately  cared-for  population.    The  staging  technique 
cannot  pinpoint  where  in  the  process  of  care,  or  if  indeed  it  is 
in  the  medical  care  system,  the  reasons  for  the  problem  lie.  However, 
it  can  comment  on  the  overall  health  of  a  population  and  the  general 
quality  of  care  available  to  them.    It  is  a  measure  of  true  outcome 
that  can  be  applied  quickly  and  objectively  and  can  be  used  in  a 
wide  range  of  comparative  studies. 

A  preliminary  analysis,  involving  5000  patients  admitted  to  a  sample 
of  hospitals  in  two  California  cities,  was  performed  to  test  the 
feasibility  of  the  staging  method.    Data  on  patients  in  18  disease 
categories  were  computer  processed  at  a  low  cost.    In  six  categories 
differences  were  found  between  beneficiaries  of  commercial  insurance 
and  Medi-Cal  and  beneficiaries  of  "other  government  programs"  (primarily 
county  welfare  patients),  the  latter  group  entering  the  hospital 
in  consistently  worse  condition. 

Future  improvements  in  the  staging  technique  include:    1)  developing 
staging  criteria  for  other  disease  conditions,  2)  testing  and  refining 
staging  criteria,  3)  establishing  baseline  standards  (including  time- 
phased  monitoring  for  significant  changes),  4)  developing  protocols 
for  tracking  back  along  the  process  of  care,  and  5)  developing  compari- 
son techniques. 
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A  system  of  auditing  outpatient  medical  records  was  developed  in 

1972  at  the  St.  Louis  Park  Medical  Center,  a  multi specialty  clinic 

in  Minneapolis,  Minnesota.    The  goal  of  this  ongoing  project  was 

to  evaluate  the  effectiveness  of  care  provided  through  the  use  of 

the  outpatient  medical  audit.    Among  other  things  they  hoped  to  involve 

many  clinic  physicians  (70%  actually)  in  choosing  diagnoses  for  the 

study,  in  developing  "Master  Criteria"  patterns,  and  in  setting  "Expected 

Performance"  standards  of  care;  to  use  nonphysician  paramedical  personnel 

as  auditors  for  chart  abstraction;  and  to  integrate  the  audit  into 

an  ongoing  professional  standards  and  education  program  at  the  Medical 

Center. 

Twenty  study  diagnoses  were  chosen  and  participating  physicians  developed 
"Master  Criteria"  patterns  for  each  of  these  diagnoses.    A  "Case 
Abstract  Form"  was  constructed  to  allow  its  use  with  data  processing 
equipment  and  an  "Audit  Manual"  was  prepared  to  help  the  nonphysician 
personnel  in  their  patient  record  abstraction.    Quarterly  reports 
consisted  of  individual  reports  to  each  physician  having  patients 
in  the  study,  summary  reports  to  each  clinic  involved,  and  summary 
reports  for  the  Medical  Center  as  a  whole. 

To  determine  the  validity  and  reliability,  93  records  were  reabstracted 
by  the  original  nonphysician  abstractor  and  one  physician  coordinator. 
Comparison  data  were  formulated  for  physician  vs  original,  physician 
vs  reaudit,  original  vs  reaudit.    There  was  substantial  investigation 
and  discussion  of  error  origination.    It  was  found  that  the  most 
consistent  defect  in  all  of  the  disease  summaries  was  the  lack  of 
record  concerning  outcome.    The  article  does  not  address  the  problem 
of  patients  with  multiple  diagnoses,  the  time  frame  of  the  study, 
or  the  feedback  from  the  physicians  regarding  the  reports  given  them. 

Conclusions:    Paramedical  personnel  can  be  used  to  abstract  medical 
information  from  the  records,  which  saves  physician  time  and  thus 
is  cost-effective.    Plans  are  to  continue  using  the  audit  system 
for  an  ongoing  evaluation  of  professional  activities. 
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The  American  Society  of  Internal  Medicine  undertook  a  study  to  develop 
a  method  for  assessing  quality  in  the  office  practice  of  internists 
and  to  see  if  this  method  is  effective  in  promoting  and  directing 
continuing  medical  education  for  the  improvement  of  the  quality  of 
care.    Participants  were  all  general  internists  from  four  states 
(New  York,  Colorado,  Washington  and  Georgia)  representing  four  regions 
of  the  country.    In  New  York  the  physicians  were  divided  into  three 
groups,  practicing  in  metropolitan,  urban  or  rural  areas.  One  hundred 
fifty  physicians  were  selected  at  random  from  volunteers.    Six  categories 
of  care  were  selected:    1)  newly  discovered  adult  diabetes  mellitus, 
2)  follow-up  of  established  diabetes  mellitus,  3)  newly  discovered 
diastolic  hypertension,  4)  follow-up  of  established  hypertension, 
5)  female  acute  urinary  tract  infection,  and  6)  periodic  health  evalua- 
tion. 

An  "exhaustive"  list,  prepared  by  the  members  of  the  Quality  Evaluation 
Committee,  was  sent  to  the  participants  for  additions,  deletions 
and  scoring  on  a  five-point  scale  from  highly  relevant  to  contra- 
indicated.  When  an  average  grade  for  an  item  was  smaller  than  the 
grand  average  for  that  category  and  region,  it  was  considered  a  poor 
indicator  of  quality.    A  validation  study  of  these  criteria  was  also 
conducted.    To  facilitate  and  assure  recording  and  processing  of 
the  patient  care  data,  a  data  collection  form  was  prepared.  The 
physicians' s  performance  was  scored  by  assigning  points  to  the  various 
items  in  the  established  process  of  care,  the  physician  receiving 
no  points  when  procedures  were  not  done  or  recorded.    Cases  with 
deviations  from  established  norms  might  be  referred  to  peer  review 
committees  for  further  study. 

It  was  demonstrated  that  physicians  of  similar  backgrounds  and  with 
minimum  interaction  agreed  very  well  in  the  establishment  of  criteria 
for  quality  care  for  common  diseases.    They  also  agreed  well  with 
expert  academicians  and  practitioners.    However,  physicians  seldom 
agreed  on  the  relative  weighting  of  each  of  the  criteria.    A  preferable 
weighting  system  would  be  one  based  on  the  demonstrated  relationship 
between  the  process  and  outcome  of  care.    The  data  collection  form, 
which  should  be  a  permanent  part  of  the  patient's  chart,  should  be 
completed  by  a  physician's  aide.    A  problem-oriented  approach  that 
demonstrates  the  logic  of  the  medical  care  process  would  be  most 
beneficial.    The  study  also  showed  that  there  is  little  correlation 
between  the  physician's  theoretical  approach  (criteria)  to  ideal 
care  and  his  actual  performance.    This  statistical  quality  control 
mechanism,  although  not  applicable  to  the  evaluation  of  individual 
cases,  can  apply  to  the  review  of  physician  profiles  and  regional 
performance  and  can  identify  physicians  who  are  over  or  under-util izers 
of  process  of  care. 
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Holloway,  Don  C. ,  Wiczai  L.  James,  Carlson,  Edgar  J.:    Evaluating  an  information 
system  for  medical  care  evaluation  studies,  Med  Care  13:329-340, 
Apr  1975  (P-D),  CA,  DC,  ES-P 

A  study  was  conducted  which  compared  a  computerized  information  system, 
the  Professional  Activity  Study  -  Medical  Audit  Program  (PAS-MAP), 
with  a  manual  system.    Although  PAS-MAP  can  also  compile  death  rates, 
index  and  cross-index  hospital  discharges,  select  patients  for  studies, 
perform  administrative  and  financial  functions,  etc.,  only  its  use 
in  medical  care  evaluation  studies  was  examined.    The  PAS-MAP  and 
manual  systems  were  compared  using  four  criteria:    1)  completeness 
(degree  to  which  requested  data  could  be  provided,  limited  to  that 
which  could  be  abstracted  from  the  medical  record),  2)  timeliness 
(degree  to  which  the  data  reach  the  study  process  in  time  to  influence 
action  -  2  months  in  this  study),  3)  operability  (ability  to  offset 
potential  human  error),  and  4)  cost. 

In  the  manual  system  the  data  in  the  medical  records  of  patients 
with  the  diagnosis  under  study  were  abstracted  onto  a  worksheet  and 
findings  were  summarized.    In  the  computerized  system  the  medical 
record  data  were  abstracted  using  PAS-MAP  abstract  forms  which  were 
then  mailed  to  the  Commission  on  Professional  and  Hospital  Activities. 
The  CPHA  provided  the  hospital  with  monthly  reports  of  the  findings. 
Although  the  study  method  guaranteed  a  100%  score  in  completeness 
for  the  manual  system,  the  PAS-MAP  scored  considerably  lower,  especially 
in  General  Practice  (18  and  20%).    Both  systems  proved  timely  but 
a  manual  system  would  have  to  augment  PAS-MAP  where  it  was  incomplete. 
Partially  because  PAS-MAP  processed  considerably  more  data  (87%  were 
care  elements  unrequested  in  this  study),  the  manual  system  protected 
more  against  human  error  than  PAS-MAP  did.    PAS-MAP  was  less  costly 
if  more  than  41%  of  the  hospitalized  patients  were  included  in  the 
studies.    After  the  criteria  were  weighted  for  importance,  the  manual 
system  was  recommended  over  PAS-MAP. 
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Hulka,  Barbara  S.,  Kupper,  Lawrence  L.,  Cassel ,  John  C. :    Practice  character- 
istics and  quality  of  primary  medical  care:    the  doctor-patient  relation- 
ship, Med  Care  13:808-820,  Oct  1975  (P-D),  CS,  PPR,  QA 

The  doctor-patient  relationship  is  an  integral  and  large  part  of 
medical  care,  yet  has  remained  immeasurable.    Three  elements  of  this 
relationship  have  been  identified:    1)  communication  (the  provider's 
ability  to  increase  patient  knowledge  and  to  get  compliance  to  a 
therapeutic  regimen);  2)  satisfaction  (patient's  attitude  toward 
medical  care  received,  the  provider,  and  the  care  system);  and  3) 
physician  awareness  of  patient  concerns.    Three  hundred  sixty-three 
pregnant  women  and  523  mothers  with  infants  were  studied  for  the 
care  of  the  pregnant  woman  and  the  care  of  the  normal  infant.  The 
49  physicians  (family  doctors,  obstetricians  and  pediatricians)  were 
selected  randomly.    Both  patient  and  physician-practice  characteristics 
were  recorded  to  determine  their  effect  on  the  primary  relationships. 

Specific  items  of  communication  between  doctor  and  patient  were  selected 
for  both  pregnancy  and  normal  infant  care.    The  physician  indicated 
which  items  were  communicated  to  the  patient,  and  the  patient  was. 
given  a  corresponding  set  of  questions  to  determine  what  information 
was  comprehended.    The  doctor  was  given  another  list  for  conditions 
of  issues  that  might  cause  anxiety  in  the  patient  and  was  instructed 
to  indicate  his  perceptions  of  the  patient's  attitudes.    The  patient 
was  given  the  same  list  but  was  asked  to  express  her  own  attitudes. 
The  satisfaction  scale  (professional  competence,  physician's  personal 
qualities,  and  cost  and  convenience)  was  a  modified  version  of  the 
Thurstone  Equal  Appearing  Interval  Technique. 

It  was  found  that  more  than  half  of  the  information  intended  to  be 
communicated  to  the  patient  was  communicated.    Patients  seemed  well 
satisfied  with  their  medical  care  in  all  three  categories.    The  doctors 
seemed  aware  of  the  patient's  concerns  60  to  85%  of  the  maximum  possible. 
A  stepwise  regression  analysis  was  used  to  review  the  relationship 
between  physician-practice  characteristics  and  doctor-patient  interaction 
dimensions.    Of  interest  among  these  findings  was  that  doctors  trained 
in  approved  residency  programs  seemed  more  skillful  in  communication; 
the  fewer  the  members  of  a  practice,  the  more  effective  the  communication; 
and  for  pregnant  women  younger  physicians  communicated  more  effectively 
than  older  physicians.    Since  this  analysis  did  not  explain  differences 
in  physican  awareness  of  patient  concerns,  it  was  thought  to  be  related 
to  the  individual  physician's  personality  and  style  of  practice. 


94 


ABSTRACTS  OF  SELECTED  ARTICLES 


Hulka,  Barbara  S.,  Zyzanski,  Steven  J.,  Cassel ,  John  C. ,  and  Thompson, 

Shirley  J.:    Scale  for  the  measurement  of  attitudes  toward  physicians 
and  primary  medical  care,  Med  Care  8:429-436,  Sept-Oct  1970  (T), 
CS,  PPR 

To  overcome  the  difficulties  of  direct  questioning,  the  Thurstone 

"Method  of  Equal  Appearing  Intervals,"  a  psychological  scaling  technique, 

was  adapted  to  the  problem  of  measuring  patient  satisfaction  with 

medical  care.    Methods  used  to  develop  the  scale  and  to  validate 

the  theoretical  basis  of  the  technique  are  presented.  Strengths 

and  limitations  are  discussed  and  a  copy  of  the  final  scale  presented. 

Studies  were  administered  to  determine  if  judges  with  differing  attitudes 

were  competent  and  able  to  score  statements  (attitudes  toward  physicians) 

in  an  objective  way.    Edwards'  "Techniques  of  Attitude  Scale  Construction" 

was  reviewed  to  develop  these  statements,  which  were  then  checked 

for  content  and  format  by  the  Educational  Testing  Service.  Professional 

competence,  personal  qualities,  and  cost/ convenience  were  the  three 

content  areas  most  frequently  discussed.    The  statements  were  edited 

prior  to  the  test  and  placed  in  random  order  using  a  table  of  random 

members.    Three  groups  of  judges  were  chosen:    physicians,  social 

workers,  and  members  of  a  women's  club.    They  were  instructed  to 

score  each  statement  from  1  to  9  (1  being  the  most  favorable).  Certain 

returns  were  excluded  for  incompleteness  and  obvious  reversal  on 

the  favorable-unfavorable  continuum.    "A  scale  value  S,  the  median 

value  from  the  distribution  of  scores,  was  calculated  separately 

for  each  statement  for  each  group  of  judges.    Likewise  a  Q  value, 

which  equaled  the  inter-quartile  range,  was  calculated  to  measure 

variation  around  the  median  score."    To  show  the  degree  of  concordance 

for  S  values  among  the  three  sets  of  judges,  correlation  coefficients 

were  found  using  the  Pearson  Product-Moment  Method--. 98  for  women's 

club  and  social  workers;  .99  for  women's  club  and  physicians;  and 

.98  for  social  workers  and  physicians.  This  seems  to  verify  the  assumption 

that  persons  with  different  characteristics  and  attitudes  can  perform 

as  reliable  judges. 

A  pretest  was  conducted  with  low  income  black  and  white  mothers  in 
a  clinic  setting.    Parallel  form  reliability  was  tested  in  the  three 
content  areas  with  the  following  coefficient  correlations:    .75  for 
personal  qualities;  .63  for  professional  competence,  and  .43  for 
cost/convenience.  The  significance  of  defining  three  unique  content 
areas  is  yet  to  be  determined.    The  statements  in  the  personal  qualities 
and  professional  competence  areas  support  the  above  assumption. 
However,  the  cost/ convenience  items  need  further  modification. 
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Hurst,  J.  Willis:    How  to  implement  the  Weed  system,  Arch  Intern  Med 
128:456-462,  Sept  1971  (T),  MR 

Based  on  experience  implementing  the  Weed  system  for  improving  medical 
records  in  three  hospitals  in  Atlanta,  three  obstacles  to  its  implementa- 
tion have  been  identified.    Because  not  all  personnel  involved  are 
highly  interested  in  the  system,  there  are  obstacles  related  to  the 
acceptance  of  a  new  system.    Even  those  who  agree  to  the  values  of 
the  new  system  may  be  hampered  by  their  old  habits.    Obstacles  related 
to  the  mechanics  of  the  system  can  be  dealt  with  in  appropriate  meetings, 
however  many  that  may  take.    Although  people  must  be  attentive  to 
the  details,  they  should  not  see  this  as  the  end  of  the  system. 
It  is  only  after  they  have  mastered  the  mechanics  that  the  true  value 
emerges . 

Perhaps  the  most  difficult  obstacles  to  overcome,  however,  are  those 
related  to  the  creation  of  an  excellent  problem  list.    An  incorrect 
or  inadequate  data  base  will  result  in  an  incorrect  problem  list. 
Using  nonphysician  assistants  and  computer  techniques,  the  physician 
must  develop  reliable  methods  to  assure  a  complete  data  base.  However, 
given  an  accurate  and  complete  data  base  the  physician  still  must 
manipulate  and  interpret  the  data,  using  disciplined  thought,  to 
arrive  at  a  good  problem  list.    A  major  objective  of  his  or  her  educa- 
tional system  was  to  develop  such  a  self-disciplined  and  logical 
thought  process.    Yet  this  thought  practice  must  continue  to  be  practiced, 
for  the  initial  plan,  orders,  progress  notes  and  discharge  notes 
are  all  based  on  the  problem  list.    Although  the  problem  list  is 
useful  to  the  physician  who  created  it  and  should  not  be  judged  by 
another  as  "right"  or  "wrong,"  a  physician  more  skillful  in  collecting 
and  interpreting  the  data  base  can  use  the  exploration  of  the  differences 
between  the  two  problem  lists  as  an  educational  tool. 

The  physician's  task  is  to  order  the  data  into  a  meaningful  diagnosis. 
The  distinction  between  a  problem  that  can  be  ordered  into  a  diagnosis 
and  that  which  is  a  more  problem  should  be  maintained.    A  system 
diagnosis  should  include  four  subdiagnoses:    etiologic,  anatomic, 
physiologic  and  functional  diagnoses.    In  essence,  the  Weed  system 
is  a  problem-solving  system. 
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Kaufman,  Theodor,  Vorzimer,  Jefferson  J.  and  Metz,  David:    The  POMR  in 
an  ED,  Hospitals  JAHA  48:59-62,  1  Nov  1974  (P-D),  MR 

In  April  1973  the  Emergency  Department  of  Beth  Israel  Hospital,  New 
York  City,  devised  and  began  using  a  slightly  altered  form  of  Weed's 
Problem-Oriented  Medical  Record.    First  is  recorded  the  usually  required 
registration  and  demographic  data.    The  same  clerk  records  the  patient's 
reason  for  coming  to  the  emergency  department  under  "Problem."  Using 
this  problem  as  a  starting  point,  the  physician  then  takes  a  history, 
performs  an  examination  and  arrives  at  an  initial  assessment.  If 
no  further  evaluation  is  necessary,  the  physician  forms  a  plan  for 
treatment.    If  further  evaluation  is  necessary,  the  physician  checks 
the  appropriate  boxes  and  upon  receiving  results  of  these  tests, 
records  his  final  diagnosis.    Then  a  plan  for  treatment  is  formulated 
as  well  as  a  plan  for  patient  education,  including  all  instructions 
and  information  discussed  with  the  patient.    The  condition  of  the 
patient  upon  discharge  is  also  noted. 

The  primary  physician  and  nurse  are  solely  responsible  for  this  first 
page;  all  others  must  use  a  progress  or  consultation  sheet.  Copies 
are  sent  to  the  follow-up  agency,  the  billing  department  and  the 
patient.    The  consultation  sheet,  if  a  consultation  is  requested, 
is  set  up  so  that  reviewing  the  record  and  a  signature  are  all  that 
is  required  of  the  consultant  unless  he  disagrees  with  the  entire 
formulation  of  the  problem,  which  he  explains  on  the  sheet.  This 
avoids  duplication  of  effort.    All  other  personnel  enter  their  progress 
notes  on  a  problem-oriented  progress  sheet  for  each  problem.  The 
staff  also  uses  an  emergency  department  flow  sheet  in  three  sections- 
vital  signs,  medication  administration  records  and  diabetes  record. 

Advantages  of  this  system  include:    1)  promoting  a  unified  record- 
keeping system  (records  are  available  to  both  the  comprehensive  health 
service  and  the  emergency  department);  2)  promoting  legible  and  complete 
notes,  including  reasoned  diagnoses  and  treatment;  3)  preventing 
careless  omissions  and  unnecessary  duplication;  4)  promoting  the 
ease  and  speed  of  auditing;  5)  providing  a  structured  audit  for  the 
utilization  review  committee.    The  only  real  difficulty  in  using 
the  POMR  is  changing  the  habits  of  the  staff. 
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Kessner,  David  M. ,  Kalk,  Carolyn  E. ,  Singer,  James:  Assessing  health 
quality  -  the  case  for  tracers,  New  Engl  J  Med  288:189-193,  1973 
(T),  CSD,  ES-P 

The  basic  premise  of  the  tracer  method  is  that  the  general  quality 
of  care  and  the  efficacy  of  the  care  system  will  be  reflected  in 
how  a  physician  or  group  of  physicians  routinely  administer  care 
for  common  disease  conditions.    A  tracer  is  a  health  problem  that 
demonstrates  how  particular  parts  of  the  health  care  system  work 
in  relation  to  one  another.    This  method  measures  both  process  and 
outcome  of  care.  Criteria  for  selecting  health  problems  as  tracers 
include:    1)  a  tracer  should  have  a  definite  functional  impact,  2) 
a  tracer  should  be  relatively  well  defined  and  easy  to  diagnose, 
3)  prevalence  rates  should  be  high  enough  to  permit  the  collection 
of  adequate  data  from  a  limited  population  sample,  4)  the  natural 
history  of  the  condition  should  vary  with  utilization  and  effectiveness 
of  medical  care,  5)  the  techniques  of  medical  management  of  the  condition 
should  be  well  defined  for  at  least  one  of  the  following  processes: 
prevention,  diagnosis,  treatment  or  rehabilitation,  and  6)  the  effects 
of  nonmedical  factors  on  the  tracer  should  be  understood. 

The  activities  of  a  health  care  system  are  divided  into  five  major 
groups:    prevention,  screening,  evaluation,  management  and  follow- 
up.  Therefore  quality  of  care  can  be  sampled  from  various  perspectives. 
To  avoid  rigidity  in  medical  practice,  the  criteria  against  which 
services  performed  can  be  compared  should  outline  minimal  care,  should 
be  pragmatic,  and  should  be  periodically  revised  and  updated.  They 
should  be  considered  broadly  applicable  to  populations  of  patients. 
The  results  of  an  evaluation  should  get  back  to  those  who  set  standards 
for  acceptable  levels  of  care.    A  tracer  evaluation  in  a  hypothetical 
situation  is  described  for  illustrative  purposes. 
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Komaroff,  Anthony  L.,  Reiffen,  B.,  and  Sherman,  Herbert:    Protocol  for 
paramedics:    a  quality  assurance  tool,  Quality  Assurance  of  Medical 
Care  PHEW  (HSM)  73-7021,  Feb  1973,  pp.  161-178  (P-D),  ES-S,  MR 

A  system  of  problem-oriented  protocols  which  directs  the  management 
of  common  illnesses  by  paramedics  is  being  developed  by  the  Ambulatory 
Care  Project,  a  joint  effort  between  Beth  Israel  Hospital  (Harvard 
Medical  School)  and  M.I.T.  Lincoln  Laboratory.    This  system  defines 
an  explicit  standard  of  care.    In  this  study  paramedics  perform  patient 
evaluation  during  return  visits  of  chronically  ill  patients.  The 
protocols  guide  the  paramedics  in  data  collection  and  in  recommending 
action  including  the  necessity  of  a  visit  with  the  doctor.    The  physicians 
choose  the  patients  for  participation  in  this  program  which  can  be 
used  to  eliminate,  in  many  cases,  the  need  for  a  visit  with  the  doctor 
or  can  be  used  as  a  pre-physician  workup.    The  protocols  are  problem- 
specific,  can  be  constructed  for  the  presence  of  more  than  one  chronic 
disease  and  direct  collection  of  different  data  on  each  patient, 
data  which  the  paramedic  can  collect  as  accurately  as  the  physician. 
The  paramedics,  "health  assistants"  in  this  study,  do  not  have  an 
educational  degree  above  the  high  school  level. 

A  preliminary  evaluation  has  found  only  11%  false  positives  and  2% 
false  negatives,  none  of  which  were  considered  "serious."    A  10% 
error  rate  in  following  the  printed  logic  format  was  found  as  well 
as  10%  "unnecessary"  referrals.    Protocol  use  does  not  degrade  disease 
control  and  may  improve  it.    There  was  a  physician  visit  saving  rate 
of  37%  which  may  be  artificially  low.    Physician  acceptance  has  been 
shown  in  three  ways:    1)  decisions  to  send  patients  home  without 
a  physician  visit  having  been  approved  in  most  cases;  2)  physicians 
are  routing  many  more  patients  through  the  system;  and  3)  physicians 
are  not  scheduling  an  earlier  return  visit  after  a  "nonvisit."  Patient 
acceptance  has  also  been  demonstrated:    1)  only  7%  have  declined 
participation;  2)  there  is  no  increase  in  "no-shows;"  3)  only  one 
patient  has  objected  to  being  sent  home  without  a  physician  visit; 
and  4)  many  close  relationships  have  formed  between  patients  and 
health  assistants.    Many  important  questions  remain  to  be  answered, 
however. 
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Lieberman,  Harry  M.:    Evaluating  the  quality  of  ambulatory  pediatric  care 
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D),  ES-P,  MA 

The  charts  of  a  random  sample  of  registered  pediatric  patients  at 
the  Watts  Health  Center  in  Los  Angeles  were  audited  in  December  1970 
to  uncover  the  weaknesses  of  current  pediatric  care  and  to  plan  for 
improvement.    The  Watts  Health  Center  is  a  family-oriented  neighborhood 
health  center  in  a  predominantly  black  low  income  community  of  about 
35,000  people  in  a  3-square-mile  area.    At  first  registration,  a 
permanent  file  is  started  for  the  patient  and  all  his  family  members. 
A  2.5  per  cent  random  sample  (244  families)  of  the  registered  population 
was  selected  and  the  chart  of  every  patient  under  17  years  of  age 
was  recorded  on  a  survey  form  and  made  available  for  hand  sorting. 
This  sample  was  found  to  be  representative  of  the  entire  registered 
pediatric  population. 

"High  quality"  pediatric  care  was  presumed  to  be  present  when  all 

of  the  following  were  recorded  in  the  patient  file:    1 )  at  least 

one  comprehensive  work-up,  2)  immunizations  appropriate  for  age, 

3)  attention  paid  to  all  abnormal  laboratory  reports,  and  4)  appropriate 

follow-up  visits  for  all  significant  clinical  conditions  detected. 

Observations  confirmed  that  the  use  of  a  baseline  medical  chart  review 
helps  to  point  out  program  areas  that  are  in  need  of  improvement. 
Specific  problems  highlighted  by  this  chart  review  include  1)  one- 
quarter  of  registered  children  are  never  brought  to  the  center,  2) 
the  majority  of  pediatric  patients  receive  only  episodic  care,  3) 
two-thirds  of  the  pediatric  charts  have  no  record  of  immunizations, 
and  half  of  the  remainder  indicate  incomplete  immunization,  4)  a 
quarter  of  the  patients  have  inadequate  follow-up  of  an  abnormal 
finding,  5)  registered  patients  who  live  in  the  more  distant  parts 
of  the  target  area  are  less  likely  to  get  care  at  the  center,  and 
6)  a  significant  backlog  of  inadequate  follow-up  exists  in  the  17,000 
active  patient  charts. 

The  author  speculates  on  the  possible  causes  of  these  deficiencies 
and  some  directions  for  improvement  and  concludes  that  life  styles 
and  other  nonmedical  factors  should  be  considered  in  the  planning 
of  preventive  and  curative  health  services. 
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Lyons,  Thomas  F.  and  Payne,  Beverly  C. :    The  relationship  of  physicians' 
medical  recording  performance  to  their  medical  care  performance, 
Med  Care  12:463-469,  May  1974  (P-D),  DC,  MR 

Because  there  are  objections  to  measuring  a  physician's  performance 
using  the  medical  record,  a  study  was  undertaken  to  find  the  relation- 
ship between  a  physician's  medical  recording  and  medical  care  performances 
by  computing  the  correlations  between  those  measures  of  quality  that 
depend  upon  the  physician's  recording  and  those  that  are  independent 
of  the  physician.    This  study  combined  nonreactive  retrospective 
measurement  with  Rosenfeld's  method  of  secondary  analysis  of  record 
entries  that  are  independent  of  the  physician. 

Data,  sampled  from  all  22  general  hospitals  in  Hawaii  in  1968,  were 
collected  on  1239  cases  in  eight  diagnostic  categories  (acute  and 
chronic  cholecystitis,  breast  cancer,  congestive  heart  disease,  transient 
ischemic  attacks,  recent  stroke,  and  acute  and  chronic  urinary  tract 
infection).    Panels  of  Hawaii  practitioners  developed  criteria  for 
optimal  hospital  services  for  each  diagnosis  and  weighted  each  criteria 
item  for  importance.    The  criteria  included  items  in  history,  physical 
examination,  laboratory,  roentgenology,  and  special  procedures  of 
therapy.    For  each  case  two  indices  were  used:    "physician  entries" 
and  "independent  entries"  (items  recorded  by  other  personnel  if  the 
physician  requested  the  recommended  action). 

In  those  diagnoses  where  both  the  physician  entry  and  independent 
entry  indices  had  high  reliabilities,  overall  they  tended  to  be  related. 
This  study  concluded  that  1)  good  recording  is  related  to  good  practice, 
and  that  2)  the  relationship  is  not  perfect  but  it  is  statistically 
significant. 
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review  in  a  functioning  PSRO,  New  Engl  J  Med  292:671-675,  27  Mar 
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In  1971  the  Utah  Professional  Review  Organization  (UPRO)  was  estab- 
lished by  the  Utah  State  Medical  Association  (USMA).    By  July,  1974, 
UPRO-USMA  became  designated  as  sponsors  of  the  Utah  PSRO.  Physician 
support  includes  1100  members  out  of  1500  eligible  physicians.  In 
the  On-Site  Concurrent  Hospital  Utilization  Review  (OSCHUR)  program, 
a  nurse  coordinator,  hired  and  trained  by  UPRO,  reviews  admissions 
covered  by  subscribing  third  parties,  including  Title  XVIII  and  Title 
XIX.    The  nurse  coordinator  screens  and  collects  data  on  all  admissions 
and  the  involvement  of  a  physician  consultant  is  required  in  only 
10%  of  the  cases.    Discussion  between  reviewing  and  attending  physicians 
resolved  the  utilization  issue  in  all  but  47  of  34,000  cases,  when 
UPRO  certification  was  withdrawn.    Reviewing  physicians  are  compensated 
for  their  time  which  amounts  to  about  1.3  minutes  per  UPRO  patient. 
The  average  workload  for  a  nurse  coordinator  is  70  patients  and  each 
coordinator  processes  about  10  admissions  per  day.    This  person  also 
completes  the  data  collection  from  the  medical  record  on  the  last 
hospital  day  and  from  this  process  assessments  are  conducted  on  selected 
diagnoses,  a  retrospective  analysis  of  aggregate  data.  Although 
there  are  advantages  to  this  type  of  quality  assessment,  it  depends 
on  effective  education  programs  and  the  inherent  lag-time  in  bahavior 
modification.    Goals  of  the  medical  processes  applied  to  a  particular 
problem,  "management  objective,"  are  used  by  UPRO  to  simplify  outcome 
assessment. 

Special  projects  investigate  particular  aspects  of  the  care  process, 
such  as  utilization  of  an  ancillary  service.    When  warranted  by  complexity 
or  volume,  electronic  data  processing  is  employed.    In  Utah,  a  Medical 
Education  Foundation,  separate  from  but  linked  to  the  peer-review 
organization,  is  used  to  coordinate  continuing  education.    UPRO  will 
validate  a  service  as  a  process  audit  criteria  if  it  leads  logically 
to  the  next  step  in  diagnosis,  directly  influences  the  selection 
of  therapy,  or  gives  direct  information  concerning  prognosis.  Evaluation 
is  dependent  on  accurate  discharge  diagnosis  data.    The  utilization 
activity  of  hospitals,  reviewing  physicians,  and  nurse  coordinators 
is  described  in  a  monthly  report.    The  weak  link  in  this  system  seems 
to  be  the  inability  to  change  physician  behavior  through  education. 
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The  feasibility  of  utilizing  the  tracer  method  was  investigated  at 
the  Pediatric  Ambulatory  Clinic  of  the  Babies  Hospital  of  the  Columbia 
Presbyterian  Medical  Center.    The  tracer  approach,  basically  a  measure- 
ment of  the  process  of  care,  uses  the  health  care  episode  of  specific 
conditions  to  provide  a  data  base  for  assessing  the  quality  of  health 
services.    Basing  acceptability  on  high  prevalence,  significant  functional 
impact,  defined  diagnosis,  standard  management  procedures,  and  influence 
of  treatment  on  the  course  of  the  illness,  iron-deficiency  anemia 
was  selected  as  the  tracer  condition  for  this  study. 

Criteria,  regarded  as  essential  elements  of  good  care  that  could 
favorably  influence  outcome,  were  chosen  when  agreed  upon  by  at  least 
80%  of  the  physician  respondents.    Two  nonphysician  abstractors  reviewed 
selected  medical  charts  using  the  accepted  explicit  criteria.  This 
review  focused  on  four  levels  of  care:    evaluation  (history,  physical 
examination,  and  laboratory  procedures),  diagnosis,  treatment  and 
follow-up.    A  panel  of  three  physicians  weighted  each  level  of  care. 
Each  chart  was  also  reviewed  by  two  of  four  attending  pediatricians 
to  implicitly  judge  as  "acceptable"  or  "unacceptable"  each  level 
of  care  as  well  as  the  overall  quality  of  care. 

Of  the  100  study  patients  for  whom  hemoglobin  determination  was  requested 
and  found  low,  58  were  tested  for  conditions  other  than  anemia. 
In  48  cases  the  hemoglobin  level  was  recognized  and  noted  as  low. 
Of  them,  34  met  minimal  criteria  in  the  evaluation  phase,  30  were 
diagnosed  with  iron  deficiency,  27  received  treatment,  26  had  follow- 
up  arranged,  and  25  kept  their  follow-up  appoinements.  Termination 
of  care  was  usually  due  to  patient  failure  to  keep  subsequent  appointments. 
The  mean  overall  score  for  the  48  recognized  cases  was  50.7  out  of 
100.    Implicit  evaluation  resulted  in  acceptable  ratings  by  both 
physicians  in  about  60%  of  the  cases  for  physical  examination,  diagnosis 
and  treatment,  50%  of  the  cases  for  laboratory  examination,  and  37.5% 
of  the  cases  for  overall  quality  of  care.    For  each  of  the  four  levels 
of  care  and  for  the  overall  score,  explicit  review  was  closely  related 
(correlations  from  .37  to  .46)  to  implicit  review.    Physician  reviewers 
spent  an  average  3.9  minutes  per  chart.    This  study,  depending  on 
the  record  to  reflect  the  care  process,  demonstrated  serious  lapses 
in  recognition  as  well  as  deficiencies  in  treatment  and  thus  encourages 
future  use  of  this  method. 
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Patrick,  Donald  L.,  Bush,  James  W.,  Chen,  Milton  M.:    Toward  an  operational 
definition  of  health,  J  Health  Social  Behavior  6-23,  14  Mar  1973 
(P-D),  ES-0 

A  prerequisite  to  the  construction  of  a  health  status  index  is  an 
operational  definition  of  function,  one  component  of  a  definition 
of  health.    This  should  reflect  the  relative  importance  of  diseases 
and  disabilities.    Measuring  the  value  judgements  that  distinguish 
illness  from  health  is  the  main  difficulty.    Prognosis,  the  patient's 
expected  transition  to  other  levels  of  function  in  time  and  the  other 
component  of  the  social  construct  of  health,  is  not  measured  in  this 
study. 

Optimum  function  is  defined  as  "conformity  to  society's  standards 
of  physical  and  mental  well-being,  including  the  performance  of  activi- 
ties usual  for  a  person's  age  and  social  role."    The  chief  purpose 
of  this  study  was  to  classify  the  deviations  from  well-being  and 
determine  the  relative  value-weights  assigned.    The  objective  disturbances 
in  role  performance  due  to  diseases  and  disabilities  were  classified 
in  three  scales:    1)  Physical  Activity  (primarily  the  ability  to 
walk);  2)  Mobility  (freedom  to  travel);  and  3)  Social  Activity  (major, 
other,  and  self-care).    Eliminating  rare  and  impossible  combinations, 
29  functional  levels  were  selected.    To  help  rate  the  relative  desira- 
bility of  these  functional  levels,  the  variables  of  age  and  symptom- 
problem  complexes  were  added.    The  latter  variable  includes  the  subject- 
ive factors  of  pain  and  discomfort.    Mental  dysfunction  and  social 
deviance  were  excluded.    Combining  29  functional  levels,  5  age  groups 
and  42  symptom-problem  complexes  required  eliminating  those  combinations 
that  are  rare  or  impossible. 

The  Thurstone  method  of  equal-appearing  intervals  was  used  by  31 
nurses  and  31  nonmedical  students  to  assign  values  to  the  randomly 
parallel  descriptions,  each  containing  an  age  group,  three  areas 
of  functional  ability,  and  a  symptom-problem  complex.    A  16-point 
scale  was  used  with  0  representing  least  desirable  or  death  and  16 
representing  most  desirable. 

Overall,  the  nurses  and  students  agreed  quite  well  on  the  relative 
desirability  of  case  descriptions,  although  the  nurses  gave  slightly 
higher  ratings.    Because  the  judges  may  have  been  reluctant  to  use 
the  extreme  ends  of  the  scale,  truncation  resulted.    Social  and  subject- 
ive factors  not  included  in  symptom-problem  complexes  also  affected 
the  assigned  values.    Due  to  insignificant  differences  between  some 
levels,  the  list  of  29  functional  levels  could  be  further  reduced 
and  then  used  for  monitoring  populations  and  planning  health  programs. 
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Members  of  the  Connecticut  Mental  Health  Center  and  of  various  departments 
of  Yale  University  collaborated  in  the  Psychiatric  Utilization  Review 
and  Evaluation  (PURE)  Project.    Five  areas  of  the  care  appraisal 
process  were  of  major  importance:    1)  development  of  screening  mechanisms 
to  select  cases  for  review,  2)  development  of  guidelines  or  explicit 
criteria  for  the  evaluation  of  care,  3)  development  of  more  adequate 
instruments  for  collecting  data,  4)  assessment  of  factors  involved 
in  the  actual  process  of  evaluation  of  care  by  the  reviewing  physicians, 
and  5)  epidemiological  surveys  and  follow-up  studies.  Utilization 
review  in  psychiatry  was  considered  to  be  much  more  complex  than 
review  in  the  general  medical  setting  because  the  relationships  between 
diagnosis  and  treatment  and  the  efficacy  of  different  methods  of 
treatment  for  specific  illness  conditions  are  not  clearly  established. 
For  this  reason  the  Yale-developed  Basic  Utilization  Review  Program 
(BURP)  was  modified  for  psychiatric  utilization  review.    The  three 
major  phases  of  the  project  were:    1)  the  development  of  criteria 
for  evaluation  and  mechanisms  for  case  selection,  2)  the  testing 
of  criteria  and  mechanisms,  and  3)  the  implementation  of  a  system 
of  patient  care  evaluation. 

The  Data  Group  performed  a  retrospective  analysis  of  the  data  available 
in  patient  records  at  the  Connecticut  Mental  Health  Center  (CMHC) 
in  order  to  provide  preliminary  material  on  actual  patterns  of  patient 
care.    To  determine  the  best  ways  to  approach  the  problem  of  utilization 
review  in  mental  health,  panels  were  formed  on  schizophrenia  (by 
traditional  diagnosis),  depression  (by  symptom),  and  adolescence 
(by  developmental  epoch),  and  intake  (by  institutional  process). 
Follow-up  studies  on  schizophrenia  and  outpatients  would  ideally 
help  to  validate  the  hypotheses  that  might  be  developed  in  those 
related  panels. 

The  project  developed  several  criteria  to  determine  adequacy  of  care, 

especially  in  the  initial  evaluation  prior  to  assignment  to  treatment. 

It  was  concluded  that  guidelines  for  evaluating  an  individual  case 

are  best  developed  along  diagnostic  or  symptom  lines.    However,  guidelines 

for  overall  program  concerns  are  best  developed  by  focusing  upon 

institutional  processes  and  broader  social  issues.    The  components 

of  this  psychiatric  utilization  review  system  include:    1)  a  computerized 

data  system,  2)  mechanisms  for  chart  selection,  3)  guidelines  for 

care  evaluation,  and  4)  indications  for  supplementary  epidemiological 

surveys  and  follow-up  studies.    Implementation  initially  should  aim 

at  reviewing  broad  patterns  of  care  with  illustrative  individual 

cases.    Future  plans  include  implementation,  modification,  and  evaluation 

of  this  project  in  four  community  mental  health  centers. 
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The  Hayward  (Calif.)  Permanente  Medical  Group  has  devised  a  quality 
review  system  that  solves  some  of  the  shortcomings  and  difficulties 
of  current  review  procedures.    All  physicians  serve  on  the  review 
committee  on  a  rotating  schedule  and  are  assisted  in  learning  this 
activity.    Outpatient  records  are  selected  by  time  and  day  from  appoint- 
ment schedules  and  inpatient  records  are  selected  by  time  of  discharge. 
The  records  are  not  selected  on  any  medically  significant  basis  (e.g., 
deviation  from  committee-established  norms)  because  this  procedure 
must  cut  across  all  lines.    Many  current  record  selection  procedures 
routinely  do  not  detect  significant  areas  of  suboptimal  care  (e.g., 
selection  by  specific  therapeutic  agent  will  not  uncover  those  cases 
in  which  that  agent  was  erroneously  omitted). 

To  preserve  confidentiality  the  record  is  first  coded,  then  checked 
by  a  medical  record  technician  for  mechanical  problems,  then  reviewed 
by  a  physician  on  the  committee  who  notes  anything  in,  or  missing 
from,  the  record  which  reveals  or  will  lead  to  poor  care  and  which 
no  one  is  likely  to  defend.    Then  charts  with  significant  deficiencies 
(one  out  of  eight  ambulatory  charts  reviewed  and  95%  of  inpatient 
records  reviewed)  are  presented  to  the  committee,  where  classification 
of  the  alleged  deficiency  as  a  deviation  must  be  unanimous.  Mild 
disagreement  will  result  in  the  chart  being  accepted  as  satisfactory, 
but  strong  disagreement  will  lead  to  a  formal  educational  program 
for  the  entire  department.    Periodically  the  findings  of  the  committee 
are  submitted  to  the  entire  department,  an  action  that  often  is  the 
only  educational  activity  necessary  for  correction. 

The  purpose  of  this  peer-oriented  system  is  to  pinpoint  suboptimal 
care,  yet  it  also  does  reveal  the  behavior  of  poor  practitioners. 
However,  it  is  everyone's  responsibility  to  correct  suboptimal  care. 
Although  often  this  review  process  is  all  that  is  necessary  for  correc- 
tive action,  sometimes  a  memorandum  or  a  change  of  systems  is  in 
order.    Changes  that  have  been  made  include:    1)  a  mechanism  for 
transferring  hospital-discovered  drug  allergies  to  the  patient's 
ambulatory  charts,  2)  a  form  for  drug  histories  that  is  now  standard 
procedure,  and  3)  a  list  of  approved  abbreviations.    The  review  procedure 
has  been  extended  to  other  members  of  the  health  team. 
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Most  current  measurements  of  health  care  are  concerned  with  the  process 
of  that  care,  the  improvement  of  which  may  or  may  not  result  in  better 
health  of  the  individual  and  the  population.    This  system  overcomes 
the  difficulties  of  establishing  objective  criteria  for  measuring 
increasing  gradients  of  positive  health  by  establishing  quantitative 
negative  indexes  of  health:    unnecessary  disease,  unnecessary  disability, 
and  unnecessary  untimely  deaths.    Their  occurrence  is  a  "sentinel 
health  event,"  a  warning  signal  that  the  quality  of  care  may  need 
to  be  improved  and  that  a  scientific  search  for  the  remediable  underlying 
causes  is  justified.    Such  a  search  should  lead  to  information  about 
the  scientific,  medical,  social,  and  personal  factors  that  could 
lead  to  better  health. 

Certain  conditions  were  selected  as  sentinel  health  events  with  the 
assumption  that  the  condition  would  have  been  prevented  or  managed 
if  everything  had  gone  well.    Although  the  physician  is  surely  not 
always  responsible  for  the  errors  that  lead  to  a  sentinel  health 
event,  his  position  in  the  health  care  system  gives  him  the  initial 
and  also  some  continuing  responsibility. 

Three  lists  of  conditions  are  presented,  prepared  by  the  authors 
in  collaboration  with  representatives  of  the  National  Center  for 
Health  Statistics,  the  Center  for  Disease  Control,  the  Veterans  Admini- 
stration, and  specialists  in  many  fields.    The  lists  are  designed  for 
international  use  and  are  selected  whenever  possible  from  the  Eighth 
Revision,  International  Classification  of  Diseases  Adapted  for  Use 
in  the  United  States  to  facilitate  comparison  with  existing  health 
tabulations.  For  each  condition  in  the  three  categories  of  unnecessary 
disease,  unnecessary  disability,  and  unnecessary  untimely  death, 
an  attempt  is  made  to  indicate  whether  it  is  preventable  or  tractable. 
The  three  lists  of  conditions  include:    1)  clear-cut  and  immediate 
use  of  index  (occurrence  of  a  single  case  would  warrant  an  investigation 
into  its  cause),  2)  limited  use  (an  increase  in  the  number  of  cases 
of  a  particular  condition  would  justify  an  investigation),  and  3) 
categories  demanding  better  definition  and  special  study  (prevention 
or  diagnosis  and  treatment  are  not  well  enough  defined). 
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In  1970  the  National  Center  for  Health  Services  Research  and  Develop- 
ment began  an  experimental  program,  the  Experimental  Medical  Care 
Review  Organization  (EMCRO),  to  develop  and  evaluate  various  objective 
and  reliable  methods  for  the  areawide  or  statewide  review  of  physicians' 
services.    Priority  was  given  to  evaluating  physician  performance, 
including  developing  better  criteria,  data-collection,  and  analysis 
techniques.    Each  of  the  10  EMCRO's  founded  in  1971  and  included 
in  this  study  had  a  potential  total  of  at  least  250  participating 
physicians  and  were  affiliated  with  state  or  county  medical  societies. 
The  relationships  between  the  EMCRO's  and  third-party  insurers  varied 
with  local  conditions. 

Most  EMCRO's  had  specialty  panels,  reviewed  by  general  practitioners, 

develop  criteria  emphasizing  the  process  of  care.    Only  a  few  outcome 

criteria  were  established.    The  goal  was  to  raise  the  average  physician's 

performance  to  a  locally  defined  optimal  level.    In  six  diagnostic 

categories,  the  EMCRO's  were  to  have  conducted  a  collaborative  validation 

study.    The  EMCRO's  obtained  their  data  from  either  insurance  claims 

forms  or  chart  abstracts  and  have  adopted  standard  definitions  of 

data  items.    Using  these  diagnostic  specific  criteria,  most  EMCRO's 

will  review  care  provided  for  entire  episodes  of  illness.    The  criteria 

development  process  and  the  data  review  itself  have  been  educationally 

beneficial.    The  EMCRO's  are  evaluating  problems  in  organization, 

data  collection,  criteria  development,  cost  and  impact.  Analysis 

of  individual  patient  care  must  be  limited  yet  done  as  a  sampling. 

The  operational  definition  for  quality  is  "the  extent  to  which  scientifically 

established  procedures  in  the  diagnosis  and  management  of  serious, 

common  and  treatable  disorders  are  properly  applied  to  patients  who 

can  benefit  from  their  application."    Although  quality  assessment 

is  the  primary  purpose  of  EMCRO,  cost  containment  is  the  immediate 

chief  goal.    EMCRO  should  also  determine  the  involvement  of  nonhealth- 

professional  representatives  of  the  public  in  this  review  process. 
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The  Health  Maintenance  Act  of  1973  stipulates  that  a  participating 
health  maintenance  organization  have  a  quality  assurance  program 
that  "stresses  health  outcome."    An  18-month  study  was  performed 
in  1972-73  by  the  George  Washington  University  Health  Plan.  The 
Quality  Assurance  Executive  Board  established  ideal  standards  and 
estimated  findings  for  diagnostic  accuracy  and  therapeutic  outcomes. 
Diagnostic  accuracy  included  two  categories  of  undesirable  outcomes: 
false  positive  (misdiagnosed)  and  false  negative  (missed  diagnoses). 
Therapeutic  outcome  was  measured  on  a  functional  activities  scale 
from  0  (no  impairment)  to  5  (death).    The  three  health  problems  studied 
include  Contraception  (measured  by  comparing  desire  for  pregnancy 
with  risk),  Depression  (measured  by  the  depression  scale  of  the  Minnesota 
Multiphasic  Personality  Inventory),  and  Hypertension  (measured  by 
using  age-specific  blood  pressure  standards  set  by  the  Board). 

Results  revealed  a  highly  unacceptable  rate  (41-64%)  of  false  negatives 
in  each  condition.    Therapeutic  outcomes  in  contraception  revealed 
a  5%  abortion  rate  and  a  5%  unwanted  but  complete  pregnancy  rate. 
Unacceptably  high  levels  of  functional  impairment  (25%  in  class  4, 
25%  in  class  3)  were  found  among  the  depressives.    A  39%  inadequately 
controlled  rate  was  found  among  the  hypertensive  patients. 

Data  collection  was  hampered  by  low  response  rates  (38-63%)  and  con- 
stant changes  in  the  HMO  subscriber  pool  due  to  changes  in  residency 
and  eligibility.    The  health  accountant  was  often  required  to  make 
difficult  assessments  of  functional  level  when  other  psychological 
or  physical  conditions  were  present.    The  Board's  high  level  of  accuracy 
in  predicting  diagnostic  accuracy  seemed  to  support  the  premise  that 
physicians  can  point  out  from  their  own  experience  those  areas  where 
diagnostic  accuracy  can  be  improved.    The  project  was  too  short  to 
determine  its  effectiveness.    Doubts  were  raised  concerning  this 
project's  ability  to  fulfill  the  intent  of  the  federal  legislation. 
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Slee9  Vergil  N.:    How  to  know  if  you  have  quality  control,  Hosp  Prog 
38-43,  Jan  1972  (T),  CSD,  CX,  DC,  ES-P,  MA,  QA 

A  hospital  must  start  with  a  proper  organization  and  qualified  physicians, 
but  it  is  patient  care  that  counts.    Hard,  objective  data  must  be 
used  to  determine  quality  and  the  essentials  of  its  control  are  standards 
development,  surveillance,  and  corrective  action.    Standards  that 
measure  the  outcome  of  care  are  used  in  only  a  few  instances.  Most 
standards  employed  deal  with  the  processes  of  care,  e.g.,  all  patients 
should  meet  minimum  standards  of  laboratory  investigation,  a  hospital- 
wide  standard  for  all  patients.    More  specific  standards  can  be  set 
by  clinic,  diagnosis,  therapy,  etc.    Standards  should  be  considered 
as  descriptive  of  patterns  of  care  and  one  should  not  be  misled  by 
an  occasional  unusual  individual  case.    Although  research  produces 
the  most  reliable  and  valid  standards,  most  standards  are  derived 
from  the  judgements  of  experts.    Statistics  will  increase  in  importance 
for  determining  standards  of  care.    For  a  hospital  or  a  group  of 
hospitals  an  expert  committee  is  responsible  for  formulating  the 
standards. 

As  a  means  of  data  collection,  individual  chart  review  is  time-consuming 
and  does  not  lend  itself  to  the  recognition  and  analysis  of  patterns 
of  care.    A  computer-assisted  data  collection  system,  such  as  the 
PAS  system,  is  essential.    With  a  proper  approach,  including  the 
training  of  doctors  and  other  hospital  employees  and  the  provision 
of  a  health  record  analyst,  there  will  be  no  manpower  problem. 

The  evaluation  must  be  followed  with  corrective  action.    The  medical 
audit  process  itself  serves  as  an  important  educational  tool.  Recommenda- 
tions should  be  made  and  attitude,  cooperation  and  coordination  should 
improve.    When  deemed  necessary,  however,  strong  intervention  should 
occur.    It  must  be  clear  that  the  evaluative  process  does  change 
practice  and  in  a  reasonable  time.    The  medical  audit  will  be  the 
heart  of  the  quality  control  process. 
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Starfield,  Barbara,  Scheff,  David:    Effectiveness  of  pediatric  care: 

the  relationship  between  processes  and  outcome,  Ped  49:547-552,  Apr 
1972  (P-D),  ES-0,  ES-P,  MA 

Diagnostic  and  therapeutic  processes  and  outcome  were  compared  in 
order  to  assess  the  effectiveness  of  care.    The  hematology  laboratory 
files  for  July  -  September  1967  of  53  children  6  months  and  older 
with  newly  discovered  low  hemoglobin  levels  were  reviewed  to  deter- 
mine the  reason  for  the  initial  hemoglobin  order  and  evidence  for 
recognition,  diagnosis,  therapy  and  follow-up  of  the  condition. 
Home  interviews  were  also  conducted  which  ascertained  the  mother's 
knowledge  of  the  condition  and  recall  of  therapy. 

Of  the  53  patients  with  low  hemoglobin  levels  only  14  (26%)  were 
recorded  as  having  their  condition  recognized,  diagnosed,  treated, 
and  followed-up.    In  24  cases  (45%)  the  lab  result  was  not  recognized 
as  indicating  low  hemoglobin.    When  ordered  as  part  of  a  routine 
check-up,  hemoglobin  results  were  recognized  only  35%  of  the  time. 
Only  23  (44%)  of  the  children  were  tentatively  or  finally  diagnosed. 
Almost  a  third  of  the  patients  (7  of  22)  who  had  therapy  initiated 
denied  receiving  any  and  4  (18%)  had  no  follow-up  plan  recorded. 
The  most  important  factor  in  patient  loss  was  a  lack  of  doctor  awareness 
of  abnormal  laboratory  findings.    Therapy  was  found  to  be  the  care 
maneuver  most  closely  associated  with  good  outcome.    With  therapy 
taking  place  follow-up  may  also  be  closely  related  to  outcome.  The 
biggest  difficulty  in  carrying  out  the  study  was  finding  and  auditing 
the  medical  records. 
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Stratmann,  William  C,  Block,  James  A.,  Brown,  Stephen  P. ,  et  al . :  A 
study  of  consumer  attitudes  about  health  care:    the  control,  cost 
and  financing  of  health  services,  Med  Care  13:659-668,  Aug  1975  (P- 
D),  CX 

A  survey  of  521  households  in  the  Rochester,  New  York,  area  was  conducted 
in  May  1973  to  examine  the  public's  opinion  of  the  control,  cost 
and  financing  of  health  care  and  to  determine  the  standards  with 
which  people  judge  various  ambulatory  care  facilities.    A  set  of 
open-ended  questions  was  used  to  elicit  opinions  of  different  care 
facilities.    The  respondent  was  also  asked  to  assign  values  to  the 
criteria  he  or  she  used  to  arrive  at  those  opinions. 

The  resultant  criteria  categories  and  weightings  were:  economic 
(3.6%),  temporal  (11.3%),  convenience  (20.6%),  sociopsychological 
(19.7%),  and  care  quality  (44.7%).    Because  this  sample  area  has 
high  levels  of  professional  quality  to  select  from,  the  care  quality 
weighting  may  be  lower  than  that  found  in  other  areas.  Generally, 
the  respondents  expressed  confidence  in  the  medical  profession. 
Forty-five  percent  preferred  planning  and  control  by  the  health  profes- 
sions, 48%  preferred  equal  involvement  of  professionals  and  consumers, 
and  7%  felt  that  the  consumer  should  plan  and  control  health  services. 
Although  economic  criteria  were  least  important  (3.6%)  in  consumer 
decisions  in  seeking  care,  64%  of  the  people  felt  that  a  cost-related 
problem  (insurance,  costs  of  services,  hospital  costs,  etc.)  was 
where  the  government  should  get  involved.    In  this  sample  area  28% 
favored  government  financing  of  health  services,  41%  favored  equal 
financing,  and  31%  favored  private  financing.    This  compares  to  37%, 
29%  and  34%  respectively  in  national  samples  in  1970  and  1972.  The 
difference  may  be  due  to  the  Republican  partisanship  in  the  Rochester 
area.    Income,  education,  and  health  status  are  inversely  proportional 
to  support  for  national  health  insurance. 
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Thompson,  Hugh  C,  Osborne,  Charles  E. :    Development  of  criteria  for  quality 
assurance  of  ambulatory  child  health  care,  Med  Care  12:807-827,  Oct 
1974  (P-D),  CSD 

In  May  1970,  the  Joint  Committee  on  Quality  Assurance  (JCQA),  with 
members  from  eight  national  professional  organizations  dealing  to 
some  degree  with  primary  care  for  children,  began  a  study,  the  first 
phase  of  which  was  to  select,  develop  and  validate  criteria  for  the 
evaluation  of  ambulatory  child  health  care.    Because  it  lends  itself 
well  to  quantification,  the  chart  audit  approach  was  used  to  collect 
data.  To  counteract  weaknesses  of  process  and  outcome  evaluations, 
the  JCQA  attempted  to  identify  process  procedures  likely  to  be  causally 
related  to  patient  outcomes  according  to  available  studies  or  the 
consensus  of  qualified  experts. 

The  first  step  was  to  select  seven  health  problems:    health  supervision 
in  various  age  brackets,  tonsillopharyngitis,  female  urinary  tract 
infection,  and  bronchial  asthma.    The  JCQA  developed  an  extensive 
list  of  process  criteria  for  each  area.    To  validate  the  relevance 
of  the  criteria  to  outcome  and  thus  acceptability  for  peer  review 
or  self-assessment,  the  list  was  rated  by  388  "expert"  academicians 
and  practitioners.    The  JCQA  then  established  a  final  list  of  138 
criteria  that  were  rated  relevant  by  at  least  85%  of  the  "expert" 
panel.    There  was  a  surprising  agreement  between  the  opinions  of 
the  "expert"  academicians  and  practitioners.    Although  the  experts 
generally  agreed  with  the  initial  JCQA  classification,  there  was 
enough  difference  between  the  initial  and  final  versions  to  suggest 
that  the  formulation  of  criteria  be  done  by  those  accustomed  to  such 
procedures.    In  the  other  phases  of  this  study  the  attitudes  of  physi- 
cians toward  the  criteria  and  the  presence  of  the  criteria  in  130 
sites  were  examined. 
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Thompson,  Hugh  C. ,  and  Osborne,  Charles  E.:    Office  records  in  the  evaluation 
of  quality  of  care,  Med  Care  14:294-314,  Apr  1976  (P-D),  MA,  MR 

A  previous  portion  of  this  study  validated  criteria  for  child  health 
supervision  (in  four  age  groups)  and  three  diseases  (tonsillopharyn- 
gitis,  bronchial  asthma,  and  female  urinary  tract  infection)  for 
relevance  to  outcome  and  suitability  for  peer  review.    The  Joint 
Committee  on  Quality  Assurance  (JCQA)  then  selected  166  physicians 
whose  office  charts  were  to  be  reviewed  to  determine  if  in  fact  these 
physicians  recorded  performance  of  these  144  criteria  and  subsequently 
to  be  able  to  judge  the  quality  of  care.    Family  physicians  and  pediatri- 
cians, divided  into  solo,  small  groups  (2-5)  and  large  groups  (6 
or  more),  were  selected.    Each  physician's  last  ten  charts  in  each 
category  prior  to  the  date  of  the  review  were  to  have  been  pulled, 
by  the  office  personnel.    Fourth-year  medical  records  administration 
students  performed  the  reviews. 

Only  two- thirds  of  the  office  personnel  followed  the  prescribed  method 
for  selecting  charts.    In  general,  the  established  criteria  were 
recorded  infrequently,  overall  about  50  per  cent.    Growth  and  development 
measurements  were  the  only  criteria  in  the  health  supervision  category 
recorded  more  than  75  per  cent  of  the  time.    Counseling  and  guidance 
were  the  least  frequently  recorded  criteria.    Physicians  agreed  with 
the  reviewers'  findings  most  of  the  time  (96-100%).  Pediatricians 
and  family  physicians  recorded  criteria  with  about  the  same  frequency, 
except  for  38  of  the  74  health  supervision  criteria  which  the  pediatri- 
cians recorded  more  often.    Physicians  in  large  groups  recorded  criteria 
more  frequently  than  those  in  solo  or  small  group  practices.  Only 
48  per  cent  of  the  physicians  felt  the  audit  gave  an  accurate  picture 
of  their  performance.    Most  of  the  rest  said  they  recorded  only  positive 
findings.    Usually  the  audit  was  completed  in  an  eight-hour  working 
day. 

Bias  may  have  been  introduced  by  selecting  physicians  most  willing 
to  cooperate  and  thus,  possibly,  those  with  better  recording  systems. 
Notwithstanding,  the  demonstrated  lack  of  recording  prevents  an  accurate 
and  meaningful  chart  audit  of  ambulatory  child  care.    The  challenge 
is  still  to  develop  and  implement  new  concepts  of  medical  recording 
keeping,  better  documenting  the  logic  of  medical  care. 
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Wagner,  Edward  H.,  Greenberg,  Robert  A.,  Imrey,  Peter  B.,  et  al.:  Influence 
of  training  and  experience  on  selecting  criteria  to  evaluate  medical 
care,  New  Engl  J  Med  294:871-876,  15  Apr  1976  (P-D),  CSD 

To  determine  the  influence  of  varying  training  and  experience  on 
the  selection  of  process  criteria  for  medical  care  evaluation,  three 
groups  of  physicians  were  sampled:    general  pediatricians,  family 
physicians,  and  pediatricians  subspecial izing  in  infectious  diseases 
(academicians).    The  indicator  condition,  associated  with  10  possible 
diagnoses,  was  "a  6-month  to  2-year-old  infant  with  recent  onset 
of  rhinorrhea  and/or  cough."    One  hundred  twenty-five  test  situations 
(symptom- sign  combination  followed  by  a  diagnostic  or  therapeutic 
action)  were  selected. 

The  questionnaire  asked  the  physician  to  rate  the  necessity  of  the 
action  in  each  test  situation  on  a  7-point  scale  from  absolutely 
necessary  to  completely  unnecessary.    Their  responses  were  grouped 
as  favoring,  opposing  or  neither  favoring  nor  opposing.    A  physician 
was  considered  to  have  favored  an  action  if  65%  or  more  individuals 
favored  and  20%  or  less  opposed  it.    If  all  three  groups  shared  the 
same  opinion  (favor,  oppose,  or  uncertain)  they  were  considered  to 
have  agreed.    Agreements  to  favor  or  oppose  an  action  established 
it  as  a  consensus  criterion.    Distribution  of  uncategorized  responses 
were  also  compared.    There  was  an  overall  54%  (524)  return  rate. 

There  was  agreement  in  74%  (93)  of  the  test  situations:    52%  favoring 
an  action,  22%  uncertain,  and  in  one  test  situation,  opposing  it. 
Thus,  53%  (66)  could  be  classified  as  consensus  criteria.    When  a 
criterion  was  applied  in  a  more  benign  situation,  it  was  also  applied 
in  more  serious  situations.    Disagreements  (26%)  were  most  prevalent 
concerning  antibiotic  use.    In  most  disagreements  the  family  physicians 
were  most  likely  to  favor  the  action.    In  infectious-diseases  pediatri- 
cians were  least  inclined  to  prescribe  antibiotics.    The  general 
pediatricians  were  the  least  demanding  concerning  history  taking, 
physician  examination,  and  diagnostic  tests.    In  general,  an  emphasis 
was  made  on  the  quality  and  completeness  of  medical  records.  However, 
most  of  the  66  consensus  criteria  have  yet  to  be  supported  by  outcome 
data. 
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Waldman,  Martin  L.:    The  medical  audit  sutdy  -  a  tool  for  quality  control, 
Hosp  Prog  82-88,  Feb  1973  (T),  CSD,  MA,  MR 

A  medical  audit,  a  hospital  quality  control  system,  is  composed 
of  a  series  of  medical  audit  studies,  the  performance  of  which  is 
educational  for  the  physician  and  points  out  needed  structural  and 
process  changes.    A  medical  audit  study  requires:    1)  criteria  develop- 
ment; 2)  care  evaluation;  3)  pinpointing  deficiencies;  4)  corrective 
action;  and  5)  evaluating  the  effectiveness  of  the  corrective  action. 
The  ten  steps  of  a  Medical  Audit  Study  Worksheet  (MASW)  are  described. 

Step  1  -  Study  Specifications:    A  group  of  patients  with  a  common 
characteristic  should  be  defined.    This  can  be  done  at  various  levels 
of  detail  from  "all  patients"  to  those  with  a  "specific  clinical" 
characteristic.    A  time  period  for  data  collection  should  be  chosen 
and  the  age  and  sex  distribution  should  be  recorded.    Step  2  -  Study 
Schedule:    Dates  should  be  set  for  adopting  criteria,  data  collection, 
study  completion.    Step  3  -  Pattern  Standards:    The  elements  of 
the  care  process,  including  investigation,  management,  therapy  and 
hospital  outcomes,  which  can  assess  the  quality  of  care  and  refer 
to  available  data,  should  be  enumerated.    For  each  of  these  elements 
a  pattern  standard,  a  percentage  which  predicts  how  often  an  element 
should  be  found  or  done  for  the  quality  of  care  to  be  excellent, 
is  established.    Step  4  -  Pattern  of  Care:    A  health  record  analyst 
must  abstract  and  display  the  pertinent  data  from  the  medical  records. 
The  practice  display  and  statistical  tabulations  from  PAS  and  MAP 
can  be  of  help.    Step  5  -  Problem?:    Deviations  that  indicate  superior 
care  should  be  publicized  and  may  result  in  revising  the  standards. 
Substandard  deviations  should  be  noted.    Step  6  -Analysis  of  Deficiencies: 
The  level  and  class  (knowledge)  or  performance)  of  the  deficiency 
should  be  determined.    Medical  knowledge  deficiencies  call  for  educational 
efforts  for  one  or  more  or  the  whole  staff.    Performance  deficiencies 
can  occur  because  the  physicians  are  not  receiving  feedback,  are 
being  interfered  with  structurally,  or  are  reluctant  to  perform 
in  an  acceptable  manner.    Step  7  -  Objectives  and  a  Timetable: 
Objectives  of  the  corrective  action,  whether  in  education  or  performance, 
should  be  enumerated  and  assigned  deadlines.    Step  8  -  Monitor  Parameters: 
To  maintain  continuity  between  studies  in  a  clinical  group,  selected 
critical  elements  should  be  monitored  from  routinely  collected  data. 
Step  9  -  Date  for  Repeat  Study:    To  ascertain  the  effectiveness 
of  the  corrective  action,  a  follow-up  study  should  be  scheduled. 
Step  10  -  Report  of  Study:    Reports  for  each  medical  audit  study 
should  be  disseminated  to  responsible  committees  and  offices  as 
well  as  to  the  whole  medical  staff. 
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At  one  level  quality  control  should  point  out  the  gross  errors  in 
medical  care.    At  the  other  level  it  should  reflect  the  excellence 
with  which  medical  care  is  carried  out.    The  definition  of  level 
of  care  implies  careful  data  collection,  problem  recognition,  explicit 
plans,  and  follow-up  and  evaluation,  not  a  rigid  list  of  "cookbook" 
instructions.    Quality  evaluation  is  possible  only  when  there  is 
prior  agreement  as  to  what  care  should  be  given  and  how  it  should 
be  recorded.    Only  that  which  can  be  realistically  expected  to  be 
achieved  should  be  audited.    Effective  audits  are  rigorous  and  well 
defined. 

The  problems  of  auditing  are  reflected  in  the  four  phases  of  medical 
action.    1)  The  data  base  (history,  physician  examination,  and  base- 
line laboratory  data)  means  asking  specific  branching  logic  questions, 
performing  specific  actions  in  the  physical  examination  and  obtaining 
certain  base-line  laboratory  data,  regardless  of  complaint,  every 
time.    2)  A  complete  list  of  problems  as  an  index  to  the  medical 
record  is  necessary  if  the  auditor  is  not  to  continue  guessing  which 
data  go  with  what  problem.    The  physician  should  know  what  record 
the  auditor  expects.    Failures  in  performance  should  be  pursued  until 
the  physicians  develop  discipline  and  techniques  for  solving  problems. 
3)  Similarly,  plans  for  each  problem  should  be  developed  so  that 
the  auditor  does  not  have  to  guess  what  treatment  goes  with  what 
problem  with  detrimental  results  to  the  audit.    4)  A  correct  follow- 
up  requires  progress  notes  titled  with  respect  to  each  problem  on 
the  problem  list.    Here  the  physician  can  explain  exactly  why  he 
did  what  he  did  and  why  he  may  have  deviated  from  the  usual  criteria. 
He  should  not  be  criticized  for  not  rigidly  sticking  to  the  criteria; 
he  must  be  audited  in  terms  of  the  logic  of  his  pursuit  of  the  problem. 
Using  structured  information  and  adequate  communication  techniques 
is  the  only  practical  way  for  the  millions  of  patients  to  receive 
the  highest  standards  of  care. 

The  article  concludes  with  the  four  premises  of  a  problem-oriented 
audit.    1)  All  the  data  after  the  initial  data  base  in  the  medical 
record  must  be  associated  with  a  specific  problem.    2)    All  the  data 
on  any  given  problem  must  be  easily  retrieved  in  sequence  and  with 
complete  currency.    3)    Conclusions  will  be  much  more  difficult  when 
there  are  concomitant  problems  in  the  same  patient.    In  these  particular 
cases  fixed  standards  of  care  do  not  apply,  and  quality  must  be  determined 
individually.    4)    The  dimensions  of  the  quality  control  problem 
alluded  to  in  premises  2  and  3  can  never  be  assessed  until  computerization 
of  the  data  is  accomplished. 
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Education  objectives  specifying  who  needs  to  learn  and  the  goals 
of  the  learning  process  can  be  identified  only  through  systematic 
investigation.    These  same  methods  can  be  used  to  evaluate  the  impact 
of  the  educational  effort.    This  study  describes  a  strategy  to  identify 
preventable  impairment  not  being  prevented  by  presently  provided 
care.    Diagnostic  outcomes  are  the  data  the  physician  needs  to  determine 
need  of  care,  therapy  and  prognosis.    Therapeutic  outcomes  are  the 
patient  health  status  following  treatment.    The  need  for  process 
study  will  be  determined  when  outcome  criteria  are  compared  to  outcome 
measurements.    Peer  judgement  is  a  practical  way  of  establishing 
outcome  criteria. 

Diagnostic  outcomes  can  be  measured  by  determining  the  proportion 
of  diagnostic  false-negatives  and  false-positives.    Therapeutic  outcomes 
can  be  measured  by  investigating  after  a  certain  period  of  time  the 
patient's  level  of  maximum  impairment:    (0)  no  impairment,  (1)  measurable 
impairment,  (2)  symptomatic,  (3)  not  a  major  life  activity,  (4)  bedridden, 
or  (5)  dead.    Process  study  would  be  warranted  if  outcome  measurements 
were  not  within  95%  confidence  intervals  as  compared  to  outcome  criteria. 
As  a  result  of  the  process  study  plans  for  action  should  be  formulated, 
and  where  warranted,  outcome  criteria  can  be  modified. 

In  one  of  eight  illustrative  studies,  the  maximum  acceptable  percentage 
of  false-negatives  for  the  diagnosis  of  urinary  tract  infections 
(UTI)  was  set  at  15%  and  that  of  false  positives  at  20%.  Outcome 
measurements  revealed  a  70%  false-negative  rate  and  a  29%  false-positive 
rate.    In  the  clearly  indicated  process  study  physicians  completed 
a  series  of  simulated  patient  problems  and  were  found  to  treat  all 
patients  complaining  of  classical  UTI  symptoms  with  antibiotics  and 
not  utilizing  urine  test  results  in  the  diagnosis.    When  meeting 
with  university  faculty  resulted  in  no  improvement  in  performance, 
it  was  decided  to  institute  routine  bacteriologic  screening  of  urine 
from  all  admissions. 

Tests  established  the  validity  of  the  group  prognostic  estimates. 
This  strategy  requires  a  focus  on  prognosis,  on  overall  patient  impair- 
ment, including  its  multiple  determinants,  and  on  continuing  education. 
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ABSTRACTS  OF  SELECTED  ARTICLES 


Williamson,  John  W. :    Outcome  assessment  for  implementing  quality  assurance 
systems,  Quality  Assurance  of  Medical  Care  PHEW  (HSM)  73-7021,  Feb 
1973,  pp.  313-328  (P-D),  ES-0 

The  article  expresses  the  feasibility  and  practicality  of  outcome 
evaluation.    Clinical  outcomes  (diagnostic  and  therapeutic)  and  economic 
outcomes  (final  direct  and  indirect  costs)  are  stressed.  Diagnostic 
outcomes  are  expressed  as  %  false  negative  and  %  false  positive. 
The  level  of  overall  functional  impairment  following  medical  care 
is  the  therapeutic  outcome  (expressed  on  a  6-level  scale  from  no 
impairment  to  death).    Depending  on  the  illness  involved,  the  time 
interval  between  care  and  therapeutic  outcome  measure  is  of  great 
importance.    The  final  outcome  is  a  product  of  one  or  more  determinants, 
each  of  which  themselves  can  be  intermediate  outcomes  of  other  processes. 
Outcome  evaluation  is  based  on  an  analysis  of  the  causal  and  correlational 
relationships  between  these  outcomes  and  determinants. 

The  three  steps  of  outcome  assessment  are:    1)  identify  the  standards 
of  maximum  acceptable  functional  impairment  following  care  for  patients 
within  a  certain  disease  category;  2)  measure  outcome  directly  with 
follow-up  interviews;  and  3)  compare  outcome  results  with  standards. 
If  correctable  deficiencies  are  found,  corrective  action  (an  educational 
program)  is  warranted;  but  if  the  deficiencies  are  found  not  to  be 
correctable,  then  adjustment  of  the  standards  is  indicated.  Two 
hypothetical  cases  are  described  followed  by  a  short  history  of  outcome 
assessment. 

The  Health  Accounting  Project,  a  practical  application  of  the  outcome 
assessment  method,  has  two  major  goals:  "1)  to  optimize  patient  health 
without  jeopardizing  resource  utilization,  and  2)  to  optimize  resource 
utilization  without  jeopardizing  patient  health."    A  staff  physician, 
devoting  10%  of  his  time,  directs  the  Quality  Assurance  program  and 
reports  directly  to  the  QA  Board  of  Directors.    A  full -time  Health 
Accountant,  with  one  or  two  years  college  education,  carries  out 
the  studies  based  on  priorities  (health  and  economic  benefits)  set 
by  a  priority  team  of  staff  physicians  and  according  to  criteria 
set  by  other  staff  physicians  augmented  by  an  outside  specialist. 
The  latter  team  also  identifies  the  source  of  discrepancies  and  may 
carry  out  the  improvement  effort.    Outcome  assessment  points  out 
when  further  process  or  structural  evaluation  is  needed  and  will 
become  increasingly  more  stressed  in  the  quality  assurance  effort 
nationwide. 


230-046  O  -  77  -  9 
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ABSTRACTS  OF  SELECTED  ARTICLES 


Wirtschafter,  David  D.  and  Mesel ,  Emmanuel:    A  strategy  for  redesigning 
the  medical  record  for  quality  assurance,  Med  Care  14:68-76,  Jan 
1976  (T),  CA,  DC,  MR 

The  medical  record  should  be  considered  a  means  to  achieve  explicit 
health  care  goals.    The  relationships  between  health  processes  and 
improved  patient  outcomes  must  be  further  researched  because  providing 
health  services  is  goal -related.    To  accomplish  this,  objective  and 
relevant  outcome  parameters  must  be  chosen.    The  record  system  must 
demonstrate  as  well  as  contribute  to  the  achievement  of  the  health 
care  goals.    Present  systems  fail  in  this  respect,  for  they  do  not 
lend  themselves  to  computer  programs  that  can  display  objective  historical 
or  physical  examination  data  demonstrating  realization  of  objectives. 

Four  steps  are  necessary  to  accomplish  this.    Selecting  problem  or 
disease  entities  will  be  determined  by  the  particular  management 
goals.    Goal  analysis  should  result  in  feasible  and  rational  disease- 
specific  objectives,  with  medical  practices  described  in  terms  of 
their  diagnostic  and  therapeutic  objectives.    To  specify  the  indicators 
of  goal  achievement,  one  must  recognize  the  parameters  by  which  attain- 
ment can  be  demonstrated,  the  conditions  for  attainment,  and  the 
standards  of  performance.    Completion  of  these  three  steps  leads 
to  a  "minimum  care  assurance  data  set,"  monitoring  outcome  through 
patient-specific  and  problem-specific  objectives.    "Minimum  care 
assurance  data  sets"  are  limited  by  being  based  upon  patient  follow- 
up  visits  and  of  course  will  change  with  advances  in  medical  knowledge. 
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ABSTRACTS  OF  SELECTED  ARTICLES 


Zuckerman,  Alan  E. ,  Starfield,  Barbara,  Hochreites,  Clare,  et  al.:  Validating 
the  content  of  pediatric  outpatient  medical  records  by  means  of  tape- 
recording  doctor-patient  encounters,  fed  56:407-411 ,  Sept  1975  (P-D), 
MR 

During  a  two-week  period  in  August  1973,  51  physician-patient  encounters 
were  tape-recorded  to  compare  information  that  was  verbally  communicated 
to  that  which  was  written  in  the  medical  record.    A  prior  examination 
of  medical  records  showed  that  the  act  of  tape-recording  had  substantially 
no  effect  on  the  frequency  with  which  information  was  written  in 
the  medical  record.    Items  examined  included  diagnoses  or  problems, 
medications  (name,  dosage,  purpose,  side  effects),  other  therapy, 
allergies,  follow-up  appointments,  diagnostic  studies,  chief  complaints, 
reason  for  seeking  care,  probable  underlying  cause  of  disease,  and 
purpose  of  follow-up.    Immediately  after  the  encounters,  the  adult 
accompanying  the  child-patient  was  interviewed  concerning  knowledge 
of  diagnosis,  medication,  other  therapy,  follow-up  appointments, 
and  diagnostic  studies.    The  presence  and  absence  of  the  above  items 
on  the  tapes  and  medical  records  was  reviewed. 

Chief  complaint,  diagnosis,  scheduled  appointments,  other  therapy, 
and  diagnostic  studies  were  all  well  recorded.    Medication  names 
were  recorded  more  often  than  told  to  patients  but  the  opposite  was 
the  case  with  dosage.    Patient  compliance,  disability,  cause  of  illness, 
reason  for  follow-up  and  allergies  were  discussed  more  often  than 
recorded.    Both  reasons  for  seeking  care  and  medication  side  effects 
were  rarely  recorded  or  discussed.    When  the  diagnoses  and  medication 
name,  function,  and  dosage  were  both  recorded  and  discussed,  patient 
knowledge  improved. 

Medical  records  may  be  expected  to  be  more  complete  at  the  study 
site  (university  medical  center)  than  in  a  general  sample  of  practices 
or  clinics.    Improved  patient  understanding  was  correlated  with  the 
concordance  of  verbal  and  written  information. 
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QUALITY  ASSURANCE  IN  AMBULATORY  CARE:    REVIEW  OF  LITERATURE 


This  section  is  a  review  of  the  articles  listed  in  the  bibliographic 
index.    Like  the  index,  it  is  not  exhaustive  but  is  expected  to  be 
representative  of  the  current  methods  and  approaches  to  the  assurance 
of  quality  in  medical  care.    As  such,  many  important  references  are  not 
included.    The  rapidly  burgeoning  literature  also  contains  much  important 
work  since  April,  1976,  which  is  not  included  in  this  review.  Indeed, 
more  detailed  and  complete  descriptions  as  well  as  more  comprehensive 
state-of-the-art  reviews  are  cited  herein  and  are  listed  and  catalogued 
in  the  accompanying  bibliography.    While  this  review  concentrates  on 
quality  assurance  in  ambulatory  medical  care,  hospital  quality  assurance 
literature  is  included  where  it  adds  to  an  understanding  of  current 
methods  and  concepts.    Although  many  of  the  articles  address  broad  quality 
assurance  topics,  the  references  are  organized  first  to  familiarize  the 
reader  with  the  definition  and  general  concepts  of  quality  assurance  and 
then  to  deal  more  specifically  with  quality  assurance  methods  and  techniques. 

QUALITY  ASSURANCE  DEFINITIONS  AND  CONCEPTS 

Many  authors  first  attempt  to  define  and  put  into  focus  the  most  basic 
concept  under  examination,  quality.    What  is  meant  by  "quality"  of  medical 
care?    Lee  and  Jones  (1933)  enumerated  the  variables  that  constitute 
quality.    In  modern  parlance  this  mul tidemensional  conceptualization  of 
the  quality  of  care  includes  "acceptability,  occurribility,  availability, 
compliance,  comprehensiveness,  coordination,  effectiveness,  and  efficiency" 
(Barr  and  Gaus,  1973).    Brook  et  al .  (Winter  1975)  have  distilled  the 
various  definitions  of  quality  and  arrived  at  two  basic  concepts:  technical 
care  and  the  art-of-care.    It  is  the  technical  aspect  of  the  quality  of 
medical  care  that  has  usually  been  measured;  but  they  feel  that  more 
emphasis  should  be  placed  on  the  development  of  the  art-of-care,  which  is 
concerned  with  the  "milieu,  manner,  and  behavior  of  the  provider  in  deliver- 
ing care  to  and  communicating  with  the  patient."    In  the  Experimental 
Medical  Care  Review  Organization  Program  the  operational  definition  of 
quality  is  "the  extent  to  which  scientifically  established  procedures  in 
the  diagnosis  and  management  of  serious,  common,  and  treatable  disorders 
are  properly  applied  to  patients  who  can  benefit  from  this  application" 
(Sanazaro  1972). 

Of  fundamental  concern  to  investigators  of  the  assessment  of  the 
quality  of  medical  care  is  the  selection  of  the  best  object  of  assessment. 
By  grouping  the  various  methods  for  evaluating  the  delivery  of  health  care, 
Donabedian  (1969)  and  Brook  et  al.  (1975)  describe  the  three  basic  conceptual 
approaches  to  quality  assessment:    process  evaluation,  outcome  evaluation, 
and  structure  evaluation.    To  date,  process  evaluation,  often  in  the  form 
of  the  medical  audit,  has  been  the  most  widely  applied  method  of  quality 
control . 

Donabedian  (1968)  feels  that  process  evaluation  is  the  most  direct 
method,  especially  when  measuring  professional  functions  such  as  diagnosis 
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and  treatment.    However,  he  considers  outcome  evaluation  the  ultimate 
validation  of  process  evaluation  standards.    Other  investigators  argue 
that  process  evaluation  has  problems  both  conceptually  and  practically. 
Process  data  are  used  by  many  to  assess  the  quality  of  care  on  the 
assumption  that  the  factors  measured  are  causally  related  to  outcome  of 
medical  care,  an  assumption  "based  on  customary  medical  practice  and  not 
on  hard  scientific  evidence."  (Brook  1974). 

According  to  Brook  and  Williams  (1975)  "structural  measurements 
are  concerned  with  the  descriptive  character" sties  of  facilities  or 
providers  (e.g.,  clinics,  hospital  beds,  physicians  per  population, 
nurses  per  physician).    Donabedian  (1968)  felt  that  the  qualifications 
of  the  staff,  organization  and  physical  structure  are  not  synonymous 
with  quality.    This  relationship  is  presumed  but  is  not  always  the  case. 


REVIEW  METHODS 

To  help  administrators  and  planners  select  a  method  of  quality 
evaluation,  various  studies  have  been  conducted  which  compare  process 
and  outcome  evaluation.    Brook  and  Appel  (1973)  compared  five  different 
methods  of  peer  review:    1)  implicit  judgment  of  process;  2)  implicit 
judgment  of  outcome;  3)  implicit  judgment  of  process  and  outcome;  4) 
explicit  judgment  of  process;  and  5)  explicit  judgment  of  outcome. 
They  found  that  the  explicit  process  judgment,  probably  the  method 
most  widely  used,  produced  the  severest  judgment:  only  1/4%  of  the 
cases  were  deemed  acceptable.    The  implicit  outcome  method,  on  the  other 
hand,  produced  a  judgment  showing  adequate  care  in  63.2%  of  the  cases. 
Starfield  and  Scheff  (1972)  compared  diagnostic  and  therapeutic  processes 
and  outcome  in  children  with  low  hemoglobin  values.    They  found  that 
therapy  was  more  closely  related  to  good  outcome  than  was  diagnosis  and 
was  unrelated  to  the  original  reason  for  the  hemoglobin  test.  Finding 
and  auditing  the  medical  record  seemed  to  be  the  biggest  problem  in 
the  study.    This  method  relates  specific  provider  and  patient  activities 
to  outcome. 

Criteria  Development 

The  selection  of  valid  and  reliable  criteria  and  standards  is  essential 
for  both  process  and  outcome  evaluation.    Rosenburg  (1975)  proposes  the 
use  of  disease  specific  criteria,  which  requires  complex  data  retrieval 
and  is  also  time-consuming.    These  disadvantages  may  hinder  the  widespread 
use  of  disease-specific  criteria. 

Williamson  (1971)  felt  that  peer  judgment  is  a  practical  way  of 
establishing  outcome  criteria.    Thompson  and  Osborne  (1974)  found  that 
academicians  and  practitioners  agreed  well  in  judging  criteria  for  use 
in  peer  review.    They  concluded,  however,  that  only  groups  accustomed 
to  the  development  of  criteria  and  with  access  to  supporting  data  attempt 
it.    Wagner  et  al.  (1967)  compared  the  criteria  selection  of  three  groups 
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of  physicians:    family  physicians,  general  pediatricians,  and  pedia- 
tricians specializing  in  infectious  diseases.    The  area  of  greatest 
disagreement  appeared  in  the  criteria  for  the  use  of  antibiotics  (family 
physicians  tended  to  use  antibiotics  more  than  the  other  groups).  However, 
most  of  the  criteria  were  agreed  upon  by  a  large,  diverse  group  of 
physicians.    Criteria  for  patient  outcome  varied  according  to  type  of 
clinical  practice. 


Process  Evaluation 

Brook  (1973)  points  out  that  "in  order  for  the  assessment  of  quality 
of  care  based  on  process  data  to  be  valid,  it  must  meet  certain  standards: 
(a)  the  procedures  for  collecting  the  information  must  produce  valid  and 
reliable  results;  (b)  the  criteria  which  are  established  must  also  be 
valid  and  reliable;  and  (c)  there  must  be  a  previously  established 
positive  relationship  between  a  high  score  on  the  process  assessment  and 
health."    Process  evaluation  measures  "what  a  provider  does  to  and  for  a 
patient  and  how  well  a  person  is  moved  through  the  health  care  system, 
either  in  a  "macro"  sense  (e.g.,  from  first  symptom,  to  seeking  care,  to 
obtaining  care)  or  in  a  "micro"  sense  (e.g.,  from  arrival  to  departure 
at  any  emergency  room  or  outpatient  clinic)  (Brook  and  Williams,  1975). 

Morehead  (1967)  described  medical  audits  concentrating  on  the  pro- 
fessional performance  of  the  physician  as  judged  by  a  clinician  surveyor 
in  the  review  of  a  medical  record.    It  was  concluded  that  when  conducting 
medical  audits,  weighting  and  scoring  components  of  medical  care  has 
advantages  when  large  numbers  of  cases  are  involved;  however,  the 
arbitrary  nature  of  such  weights  is  difficult  to  defend. 

Process  evaluation  has  taken  many  different  forms.    Lieberman  (1974) 
performed  a  review  of  a  random  sample  of  pediatric  charts  and  from  this 
was  able  to  uncover  deficiencies  in  care  and  plan  corrective  action. 
He  concluded  that  patient  lifestyles  and  nonmedical  concerns  must  be 
considered  in  planning  health  services.    Rubin  (1973)  selected  outpatient 
records  by  appointment  time  and  selected  inpatient  records  by  discharge 
time.    Physicians  noted  in  the  records  deviations  that  might  lead  to 
poor  care.    Corrective  action  then  could  be  planned,  although  the  review 
itself  may  be  all  the  action  that  is  necessary  for  improvement.  Rubin 
found  this  evaluation  method  to  be  very  well  accepted  at  the  Hayward 
(Calif.)  Permanente  Medical  Group.    Gonnella  et  al .  (1970)  discovered 
discrepancies  in  health  care  by  documenting  the  medical  status  of  patients 
before  they  were  accepted  in  the  general  medical  clinic. 

Waldman  (1973)  described  the  medical  audit,  a  hospital  quality 
control  system,  as  composed  of  a  series  of  discrete  medical  audit  studies, 
the  performance  of  which  is  educational  for  physicians  and  points  out 
the  needed  changes  in  the  process  and  structure  of  medical  care.  The 
medical  audit  study  requires:    1)  criteria  development;  2)  evaluation 
of  health  care;  3)  pinpointing  of  deficiencies;  4)  corrective  action; 
and  5)  evaluation  of  the  effectiveness  of  that  corrective  action.  Using 
a  medical  audit,  Lyons  and  Payne  (February,  1974)  found  that  elderly  and 
younger  patients  were  treated  similarly  by  physicians.    The  medical  audit 
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can  readily  be  adapted  to  evaluation  of  the  medical  care  of  ambulatory 
patients.    Hanson  and  Kraus  (1973)  found  that  in  performing  a  medical  audit 
nonphysicians  agreed  with  themselves  90%  of  the  time  and  with  a  physician 
master  audit  85%  of  the  time.    In  evaluating  health  care  in  the  emergency 
room,  Brook  and  Stevenson  (1970)  discovered  that  only  one  quarter  of  the 
patients  received  effective  care.    They  felt  that  follow-up  outcome 
evaluation  studies  were  needed. 

Levine  et  al.  (1976) describe  the  use  of  new  "health  practitioners" 
in  the  Columbia  Medical  Plan.    These  para-professionals  are  providing 
more  management  of  acute  conditions.    They  found  that  the  new  health 
practitioners  "are  providing  care,  within  their  areas  of  responsibility, 
of  comparable  quality  to  that  delivered  by  physicians."    Sackett  (1974) 
focuses  on  outcome  and  suggests  that  not  only  is  design  and  execution  of 
randomized  clinical  trials  of  innovations  in  the  delivery  of  broad 
categories  of  clinical  services  possible,  but  that  the  outcomes  provide  a 
solid  base  for  further  study  of  the  introduction  of  nurse  practitioners 
as  providers  of  primary  clinical  care.    Spector  et  al .  (1975)  discovered 
that  when  patients  were  referred  to  nurses  by  physicians,  the  nurses 
provided  adequate  care  for  most  of  the  patients  and  there  was  a  decrease 
in  physician  visits.    However,  this  was  not  cost-effective  because  a  cost 
per  patient  increase  developed  due  to  more  visits  with  the  nurses. 

Protocols  were  used  to  collect  data  on  return  visits  of  chronically 
ill  patients  in  Boston.    Komaroff  (1973)  studied  the  use  of  these  protocols 
by  "health  assistants"  who  had  no  educational  degree  above  the  high  school 
level  and  found  positive  preliminary  results:    1)  a  low  error  rate;  2)  signs 
that  show  protocols  may  improve  care;  3)  physician-visit  savings  of  37%; 
4)  physician  acceptance;  and  5)  patient  acceptance. 

Quality  assessment  has  also  entered  the  field  of  psychiatry.  Riedel 
et  al .  (1972)  feel  that  the  "state-of-the-art"  of  psychiatry  has  developed 
to  the  point  where  information  systems,  even  computers,  can  be  used  to 
find  in  the  medical  record  deviations  from  the  norms  of  usual  care.  Huber 
et  al.  (1974)  state  that  with  effort  and  creativity,  a  computerized  screen- 
ing model  can  be  developed  for  any  diagnostic  group,  even  for  psychiatric 
utilization  review. 


Outcome  Evaluation 

According  to  Gonnella  and  Zeleznik  (1974)  outcome  evaluation  should 
include  the  contributions  of  the  physician,  the  hospital  or  facility,  the 
patient,  and  the  social  and  physical  environment. 

Blum  (1974)  states  that  a  health  care  institution  should  use  more 
than  one  or  two  types  of  quality  of  care  evaluation  because  many  factors 
inside  and  outside  the  institution  affect  the  outcome  of  a  health  care 
encounter  and  each  type  of  evaluation  has  its  own  biases  and  deficiencies. 
He  believes  the  most  effective  quality-promoting  tools  are  past-focused 
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analysis  of  major  conditions  present  at  entry  to  care  and  prospective 
outcome  analysis  of  those  conditions  after  treatment  by  the  system.  He 
also  supports  the  use  of  tracers  in  relating  all  types  of  evaluation  of 
quality  of  health  care.    Sanazaro  and  Williamson  (1970)  define  and 
classify  end  results  of  patient  care  in  relation  to  and  along  with 
"critical"  physician  actions,  which  are  those  defined  as  having  one  or 
two  specific  beneficial  or  detrimental  effects  on  a  patient.  These 
classifications  contribute  to  the  development  of  objective  criteria  for 
effective  physician  performance.    Rutstein  et  al .  (1976)  conceptualized 
the  implementation  of  an  outcome  evaluation  method  that  "counts  cases  of 
unnecessary  disease  and  disability  and  unnecessary  untimely  deaths." 
For  some  diseases  the  occurrence  of  one  case  or  for  others  an  occurrence 
rate  increase  signals  the  need  for  an  investigation  of  the  causes. 

Gordis  and  Markowitz  (1971)  found  that  for  completeness  of  immunization, 
utilization  of  services,  morbidity,  and  mortality  of  infants,  there  was 
no  difference  between  Comprehensive  Care  and  Traditional  Care.    The  same 
results  were  found  for  compliance  with  a  daily  oral  penicillin  regimen  for 
rheumatic  fever.    Barr  and  Gaus  (1973)  suggested  a  population-based 
approach  to  quality  assessment.    They  measured  three  major  dimensions  of 
quality  assessment:    accessibility  (whether  patients  use  a  service  or 
not),  effectiveness  (problem  status  as  perceived  by  the  patients),  and 
efficiency  (average  unit  costs).    In  a  Health  Maintenance  Organization 
Schroeder  and  Donaldson  (1976)  compared  diagnostic  accuracy  and  therapeutic 
outcomes  with  ideal  standards.    They  found  considerable  underdiagnosis 
(false  negatives)  and  unacceptable  therapeutic  outcomes  for  depression 
and  hypertension. 

Kessner  et  al .  (1973)  applied  the  tracer  concept  to  health  care 
evaluation.    By  following  the  process  of  care  and  the  outcome  of  specific 
conditions  (tracers)  one  can  evaluate  the  quality  of  a  health  care  system. 
They  established  the  criteria  for  selecting  these  tracers:    1)  A  tracer 
should  have  a  definite  functional  impact;  2)  A  tracer  should  be  relatively 
well  defined  and  easy  to  diagnose;  3)  Prevalence  rates  should  be  high 
enough  to  permit  the  collection  of  adequate  data  from  a  limited  population 
sample;  4)  The  natural  history  of  the  condition  should  vary  with  utilization 
and  effectiveness  of  medical  care;  5)  The  techniques  of  medical  management 
of  the  condition  should  be  well  defined  for  at  least  one  of  the  following 
processes:    prevention,  diagnosis,  treatment  or  rehabilitation;  and  6) 
the  effects  of  nonmedical  factors  on  the  tracer  should  be  understood. 

Using  iron  deficiency  anemia  as  a  tracer  condition,  Novick  et  al. 
(1976)  found  weighted  explicit  scores  of  nonphysician  abstractors  to 
be  close  to  results  of  implicit  review  by  a  panel  of  physicians.  Serious 
lapses  in  the  recognition  of  this  condition  were  uncovered  as  well  as 
deficiencies  in  treatment.    These  results  seemed  to  encourage  the  use 
of  the  tracer  method  in  health  care  evaluation. 

Williamson  (1969,  1971)  has  devised  the  ABCD  strategy  which  attempts 
to  determine  "who  needs  to  learn  what  to  effect  the  greatest  improvement 
in  the  health  status  of  a  defined  population."    In  this  strategy  evaluation 
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of  diagnostic  and  therapeutic  outcomes  can  lead  to  changes  and  improvements 
in  the  diagnostic  and  therapeutic  processes.    In  Williamson's  words 
this  method  is  an  "analysis  of  management  outcome  to  determine  the  direc- 
tion and  extent  of  subsequent  study  or  alteration  of  the  management 
process  to  produce  the  most  health  improvement."    Diagnostic  outcomes 
are  determined  by  measuring  the  proportion  of  diagnostic  false-negatives 
and  false-positives  (Williamson  1971).    Final  outcome  is  the  product  of 
one  or  more  determinants,  each  of  which  themselves  can  be  intermediate 
outcomes  of  other  processes.    The  relationship,  both  causal  and  correlational, 
between  these  outcomes  and  determinants  defines  outcome  evaluation. 
Williamson  (1973)  developed  the  Health  Accounting  Project  to  demonstrate 
the  feasibility  and  practicality  of  outcome  evaluation. 

Gonnella  et  al.  (1975,  1976)  popularized  the  staging  concept  of  out- 
come evaluation  whose  basic  premise  is  that  "the  seriousness  of  a  patient's 
condition  at  some  point  in  the  treatment  process  is  a  good  indicator 
of  the  outcome  of  the  previous  parts  of  the  process."    The  purpose  of 
the  staging  concept  is  to  identify  "groups  or  clusters  of  patients  that 
require  common  treatment  procedures  and  services  and  have  similar  expected 
outcomes."    The  stages  themselves  are  the  different  levels  of  disease 
severity. 


Health  Status 

Gilson  et  al.  (1975)  developed  the  Sickness  Impact  Profile  (SIP) 
which  is  a  "behaviorally  based  measure  of  sickness  related  dysfunction." 
Preliminary  tests  showed  evidence  of  reliability  and  validity.  They 
found  the  SIP  to  be  more  appropriate  and  sensitive  than  other  outcome 
evaluation  methods.    Pollard  et  al .  (1976) found  "evidence  for  the  feasibil- 
ity of  collecting  reliable  data  using  the  SIP."    Bergner  et  al .  (1976) 
demonstrated  the  validity  of  SIP.    They  found  the  correlation  between 
validating  criteria  and  SIP  scores  to  be  high.    Patrick  et  al.  (1973) 
developed  a  list  of  29  functional  status  levels  which  classify  the  relative 
value-weights  of  deviations  from  well-being.    At  the  Kenny  Rehabilitation 
Institute  Ellwood  (1966)  developed  a  self-care  scale  which  rated  six 
daily  living  activity  categories  (bed  activities,  transfers,  locomotion, 
personal  hygiene,  dressing,  and  feeding)  to  measure  patient  improvement. 
Katz  et  al .  (1970)  have  since  1963  continued  to  develop  the  Index  of 
Interdependence  in  Activities  of  Daily  Living  (Index  of  ADL)  which  is  of 
major  importance  in  connection  with  the  dynamics  of  disability  in  the 
aging  process.    With  regard  to  the  chronically  ill,  it  is  equally  valuable 
in  objective  evaluation.    Shapiro  (1966)  felt  that  despite  the  hazards  of 
observational  research,  more  outcome  evaluation  studies  are  needed. 


Medical  Records 

Lyons  and  Payne  (May,  1974)  concluded  that  good  recording  is  related 
to  good  practice  and  that  the  relationship,  although  far  from  perfect,  is 
statistically  significant.    However,  Fessel  and  Von  Brunt  (1972)  revealed 
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conflicting  information.    They  found  that  despite  differences  in  recording 
useful  information,  different  hospitals  correctly  diagnosed  appendicitis  a 
similar  percentage  of  times.    Also,  for  myocardial  infarction  there  was 
no  correlation  between  recorded  data  and  the  outcome  after  discharge. 
They  felt  that  a  chart  review  of  the  process  of  care  may  be  misleading. 

Wirtshafter  and  Mesel  (1976)  felt  that  the  medical  record  should 
be  modified  according  to  the  information  system  design  theory  to  express 
the  relationship  between  process  and  outcome  and  to  facilitate  corrective 
action.    They  developed  a  "minimum  care  assurance  data  set"  that 
monitors  the  achievement  of  patient-specific  and  problem-specific  objectives. 
Using  prevalidated  criteria,  Thompson  and  Osborne  (1976)  found  that  a 
demonstrated  lack  of  recording  prevents  an  accurate  and  meaningful  chart 
audit  of  ambulatory  child  care,  and  they  called  for  improved  documentation 
of  medical  care.    Zuckerman  et  al .  (1975)  tape-recorded  physician-patient 
encounters  and  compared  that  information  with  the  data  in  the  medical 
record.    They  found  that  when  the  diagnosis  and  the  name,  function,  and 
dosage  of  medication  were  both  recorded  and  discussed,  patient  knowledge 
improved.    Patient  understanding  was  correlated  with  the  concordance  of 
verbal  and  written  information. 

A  variation  of  the  standard  medical  record  is  Weed's  problem  oriented 
medical  record  (POMR).    Weed  (1974)  feels  that  quality  evaluation  is 
possible  only  when  there  is  prior  agreement  as  to  what  care  should  be 
given  and  how  it  should  be  recorded.    The  physician  should  be  audited  in 
terms  of  the  logic  of  his  pursuit  of  the  problem.    Margolis  et  al .  (1973) 
found  that  Weed's  problem  oriented  record  can  measure  the  level  of  data 
collection,  data  recording,  and  problem  solving  skills.    Skills  can  be 
taught  by  grading  the  POMR. 

Using  a  slightly  altered  form  of  Weed's  POMR,  Kaufman  et  al .  (1974) 
found  several  advantages  to  this  system:    1)  promoting  a  unified  record- 
keeping system;  2)  promoting  legible  and  complete  notes,  including  reasoned 
diagnosis  and  therapy;  3)  preventing  careless  omission  and  unnessary 
duplication;  4)  promoting  the  ease  and  speed  of  auditing;  and  5)  providing 
a  structured  audit  for  utilization  review.    Hurst  (1971)  viewed  the 
Weed  system  as  a  problem  solving  system  but  discovered  three  obstacles 
to  implementing  it  to  improve  the  medical  record:    1)  acceptance  of 
a  new  system;  2)  the  mechanics  of  the  system  and  dealing  with  details; 
and  3)  creation  of  an  excellent  problem  list  to  insure  a  complete  data 
base.    To  Weed's  problem-oriented  system,  Hertz  et  al .  (1976)  added  a 
"Health  Care  Plan"  which  discusses  present  problems  and  possible  problems 
and  enumerates  possible  solutions.    This  addition  will  increase  patient- 
provider  communication  and  patient  involvement  in  solving  the  problems. 

Among  the  computerized  medical  records  systems  is  the  Professional 
Activity  Study-Medical  Audit  Program  (PAS-MAP)  which  Holloway  et  al. 
(1975)  compared  with  a  manual  system  for  quality  evaluation.    The  PAS-MAP 
system  was  found  1)  to  be  loss  costly  if  over  41%  of  patients  are 
included;  2)  to  be  as  timely  as  the  manual  system;  3)  to  provide  less 
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data  requested  by  physicians;  and  4)  to  be  more  susceptible  to  human 
error.    In  this  case  the  manual  system  was  retained. 

Of  concern  in  all  medical  record  systems  is  the  issue  of  confiden- 
tiality.   Curran  et  al .  (1969)  found  only  a  general  outline  of  the  legal 
protection  of  the  "right  to  privacy."    They  felt  that  every  centralized 
health  data  system  should  develop  rules  and  regulations  specifying 
conditions  of  information  release  as  well  as  criminal  penalties  for 
violations.    Major  contributors  and  users  should  make  up  a  large  part  of 
the  system's  policy-making  body. 


PATIENT  SATISFACTION 

The  relationship  between  the  providers  and  the  consumers  of  medical 
care  is  another  important  aspect  of  quality  assurance.    Hulka  et  al . 
(1971,  October  1975)  felt  that  patient  attitudes  and  the  physician's 
awareness  of  those  attitudes  are  important  in  the  evaluation  of  health 
care.    They  describe  the  three  elements  of  the  provider-patient  relation- 
ship:   1)  communication  (the  provider's  ability  to  increase  patient  knowledge 
and  get  compliance  with  therapy);  2)  satisfaction  (the  patient's  attitudes 
toward  the  care  received,  the  provider,  and  the  care  system);  and  3) 
physician  awareness  of  patient  concerns.    They  devised  a  method  of  measuring 
a  physician's  awareness  of  the  patient's  concerns  and  felt  that  the 
differences  found  between  various  physicians  were  related  to  individual 
personality  and  style. 

Ware  and  Snyder  (1975)  developed  index  scores  to  measure  consumer 
satisfaction.    They  arrived  at  four  major  dimensions  of  patient  attitude: 
physician  conduct,  availability  of  services,  continuity-convenience,  and 
accessibility  (cost,  payment  systems).    Hulka  et  al .  (1970)  used  the 
Thurstone  "Method  of  Equal  Appearing  Intervals"  to  establish  a  scale  to 
measure  patient  satisfaction  or  dissatisfaction.    Stratman  (July  1975)  exam- 
ined patient  perceptions  that  influence  the  selection  of  ambulatory  care 
sites  and  found  that  when  other  sources  of  care  are  not  available,  patients 
will  use  the  emergency  room  (Stratman,  December  1975).    Hulka  et  al . 
(August  1975)  found  that  consumers  were  satisfied  with  the  competence  of 
health  care  professionals  but  had  less  high  regard  for  accessibility,  cost 
and  convenience  of  health  services.    Black  women  without  a  regular  source 
of  care  were  the  most  negative.    Stratman  et  al .  (August  1975)  sampled 
households  in  Rochester,  N.  Y.,  and  found  general  confidence  in  the  medical 
profession.    About  half  of  the  sample  felt  that  there  should  be  equal 
involvement  of  professionals  and  consumers  in  the  planning  and  control  of 
health  services. 

At  the  University  of  Vermont  Bouchard  (1973)  had  positive  experiences 
sending  their  own  medical  records  to  the  patients  for  audit,  such  as: 
1)  better  patient  understanding;  2)  improved  motivation  for  complying  with 
the  therapy  regimen  and  required  life-style  changes;  3)  less  patient  anxiety. 
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CORRECTIVE  ACTION 


Although  a  major  object  of  health  care  evaluation  is  the  measurement 
of  care  and  the  discovery  of  deficiencies,  action  taken  to  correct  the 
problems  found  is  of  equal  importance.    Slee  (1972)  says  that  corrective 
action  should  improve  attitudes,  cooperation,  and  coordination,  but  that 
strong  intervention  should  be  used  when  deemed  necessary.    An  evaluation 
study  should  change  practice  habits  in  a  reasonable  amount  of  time. 
Pozen  and  Bonnet  (1976)  found  that  increased  educational  and  administrative 
intervention  improved  attitudes  and  resulted  in  better  clinic  functioning. 
However,  there  was  no  change  in  the  process  or  outcome  of  care.  Guptill 
and  Graham  (1976)  discovered  that  despite  the  assumption  that  group 
practice  leads  to  more  continuing  education,  involvement  in  various 
types  of  continuing  medical  education  was  similar  for  solo  practitioners 
and  those  in  group  practice. 

Gonnella  et  al .  (1970)  found  that  objective  examinations  showed  that 
physicians  knew  more  than  they  practiced.    It  was  concluded  that  standard 
continuing  educational  methods  will  not  translate  knowledge  into  action. 
Brook  et  al.  (October  1975)  also  advised  resisting  standard  continuing 
education.    Dutton  et  al .  (1974)  showed  that  a  problem-oriented  audit 
with  protocols  may  evaluate  the  total  process  of  care  of  specific 
components  and  that  the  continuing  feedback  is  educational  for  both 
clinicians  and  administrators.    Morehead  (1967)  felt  that  medical  audit 
in  the  Health  Insurance  Plan  of  Greater  New  York  and  the  Teamsters' 
studies  had  significant  impact  on  record  systems,  organization  of  clinical 
departments,  policies  relating  to  patient  care,  hospital  organizations, 
and  physician  qualifications. 

Brown  and  Uhl  (1970)  developed  the  "bi-cycle"  concept  of  continuing 
medical  education.    The  traditional  teacher  or  planner-oriented  approach 
is  not  satisfactory.    The  bi-cycle  concept  directly  relates  patient  care 
(outer  cycle)  to  education  (inner  cycle).    Care  deficiencies  are  found 
through  a  comparison  with  criteria  for  ideal  management  of  patients, 
and  the  closing  of  these  gaps  is  the  objective  of  education.    The  effective- 
ness of  the  corrective  action  is  determined  by  a  re-audit.    Using  this 
method,  Brown  and  Uhl  found  that  the  appropriate  use  of  antibiotics 
at  the  Chestnut  Hill  Hospital  went  from  35%  to  60%.    Two  innovative 
programs  are  described  and  it  is  concluded  that  basing  accreditation  on 
standard  continuing  educational  methods  should  not  rule  out  experimental 
programs. 

Buck  and  White  (1974)  discovered  that  in  a  prepaid  plan  (the  San 
Joaquin  Foundation  for  Medical  Care)  adjustment  of  billing  claims  in  a 
utilization  control  system  led  to  decreases  in  the  number  of  some  services 
claimed,  and  that  the  cost  of  peer  review  was  less  than  the  dollar  amount 
of  the  retrospective  adjustment. 

The  authors  cited  above  describe  the  present  approaches  to  the 
attempt  to  ensure  accurate  and  reliable  evaluation  of  the  quality  of 
medical  care  given  to  ambulatory  patients.    The  concepts,  methods,  and 
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implementation  of  quality  assurance  are  currently  in  a  rapidly  evolving 
state,  and  literature  documenting  quality  assurance  research  and 
progress  is  becoming  more  abundant. 
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